-- Logs begin at Mon 2022-07-25 04:49:34 UTC, end at Mon 2022-07-25 06:20:09 UTC. -- Jul 25 05:39:00.486368 dv-cinder-850831-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 25 05:39:00.501658 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 25 05:39:00.505826 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 25 05:39:00 2022] *** Jul 25 05:39:00.505826 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 25 05:39:00.505826 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 25 05:39:00.505826 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: nodename: dv-cinder-850831-1-iscsi Jul 25 05:39:00.505826 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: machine: x86_64 Jul 25 05:39:00.505826 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: clock source: unix Jul 25 05:39:00.505826 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: pcre jit disabled Jul 25 05:39:00.505826 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: detected number of CPU cores: 4 Jul 25 05:39:00.505826 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: current working directory: / Jul 25 05:39:00.505826 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: detected binary path: /usr/bin/uwsgi-core Jul 25 05:39:00.506530 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: your processes number limit is 47680 Jul 25 05:39:00.506530 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: your memory page size is 4096 bytes Jul 25 05:39:00.506530 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: detected max file descriptor number: 2048 Jul 25 05:39:00.506530 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: lock engine: pthread robust mutexes Jul 25 05:39:00.506530 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: thunder lock: enabled Jul 25 05:39:00.506530 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 25 05:39:00.506530 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 25 05:39:00.549759 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: Python main interpreter initialized at 0x5648864b99c0 Jul 25 05:39:00.549759 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: python threads support enabled Jul 25 05:39:00.549759 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: your server socket listen backlog is limited to 100 connections Jul 25 05:39:00.549759 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: your mercy for graceful operations on workers is 90 seconds Jul 25 05:39:00.550072 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: mapped 403077 bytes (393 KB) for 2 cores Jul 25 05:39:00.550150 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: *** Operational MODE: preforking *** Jul 25 05:39:00.550417 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: *** uWSGI is running in multiple interpreter mode *** Jul 25 05:39:00.550417 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: spawned uWSGI master process (pid: 91969) Jul 25 05:39:00.550650 dv-cinder-850831-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 25 05:39:00.551975 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: spawned uWSGI worker 1 (pid: 91980, cores: 1) Jul 25 05:39:00.551975 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: spawned uWSGI worker 2 (pid: 91981, cores: 1) Jul 25 05:39:00.551975 dv-cinder-850831-1-iscsi devstack@keystone.service[91969]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 25 05:39:05.997734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:39:05.997734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: from cryptography.utils import int_from_bytes Jul 25 05:39:05.999044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:39:05.999044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: from cryptography.utils import int_from_bytes Jul 25 05:39:06.047575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 05:39:06.047779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 05:39:06.047970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] command line args: [] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 05:39:06.048234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 05:39:06.048408 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ================================================================================ {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 05:39:06.048656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] admin_token = **** {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.048899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] config_dir = [] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.049111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.049302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] config_source = [] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.049492 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] debug = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.049711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.050913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.050913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.050913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.050913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.051176 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] list_limit = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.051236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] log_config_append = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.051420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.051608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] log_dir = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.051793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] log_file = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.052046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.052302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.052377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:39:06.052377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: from cryptography.utils import int_from_bytes Jul 25 05:39:06.052837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.052907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:39:06.052907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: from cryptography.utils import int_from_bytes Jul 25 05:39:06.053018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.053018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.053139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.053348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.053588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.053819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.054060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.054311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.054572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.054839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.055092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.055385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.055891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.056082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] publish_errors = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.056353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.056575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.056764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.057078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.057322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.057622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] standard_threads = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.057864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.058171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.058357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.058576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] use_journal = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.058764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] use_json = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.058959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] use_stderr = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.059144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] use_syslog = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.059439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.059705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.059990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.060819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.061129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.061396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.061684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.061943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.062229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.062511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.062809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.063071 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.063369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.063624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.063910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.064166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.064461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.064719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.064980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.065243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.065505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.065751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.066071 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.066366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.066697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.066946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.067211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.067452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.067746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.067993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.068259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.068518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.068814 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.069056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.069335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.069587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.069825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.070075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.070306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.070570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.070808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.071037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.071273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.071502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.071732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.071968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.072215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.072445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.072679 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.072908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.073144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.073379 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.073627 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.073855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.074090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.074324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.074579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.074815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.075047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.075276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.075522 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.075778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.076023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.076242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.076499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.076734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.076961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.077193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.077426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.077691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.077927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.078171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.078412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.078668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.078900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.079132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.079374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.079606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.079838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.080076 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.080308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.080542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.080773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.081002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.081236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.081491 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.081726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.081962 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.082195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.082433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.082689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.082920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.083151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.083386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.083614 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.083843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.084076 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.084304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.084542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.084777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.085023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.085256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.085507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.085764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.086000 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.086229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] auth.external = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.086456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.086770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.087001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.087230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] auth.password = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.087459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] auth.token = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.087694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.087927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.088169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.088396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.088640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.088883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.089111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.089344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] credential.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.089585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.089830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.090073 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.090311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.090567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.090803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.091066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.091324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.091582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.091830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.092117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.092362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.092622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.092860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.093115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.093348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] federation.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.093587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.093831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.094074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.094303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.094567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.094805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.095057 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.095288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.095540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.095773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.096016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.096251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.096493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.096737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.096971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.097204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.097562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.097796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.098033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.098281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.098553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.098788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.099019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.099097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 05:39:06.099250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.099339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 05:39:06.099466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] command line args: [] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 05:39:06.099545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.099726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.099804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 05:39:06.099927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ================================================================================ {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 05:39:06.100013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.100146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] admin_token = **** {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.100228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.100405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] config_dir = [] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.100486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.100597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.100701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.100782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] config_source = [] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.100937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.101024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] debug = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.101151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.101265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.101391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.101473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.101573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.101653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.101828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.101910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.102058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.102137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.102297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.102388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] list_limit = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.102490 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] log_config_append = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.102567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.102668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.102799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.102891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] log_dir = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.103014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] log_file = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.103106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.103252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.103333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.103494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.103573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.103728 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.103889 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.103967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.104040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.104144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.104237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.104324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.104401 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.104556 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.104647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.104804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.104885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.105038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.105117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.105272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.105355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.105506 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.105586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.105737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.105816 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.105971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.106054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.106216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.106295 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.106457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.106564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.106708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.106936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.107597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.107678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] publish_errors = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.107761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.107916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.107997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.108148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.108235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.108311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.108393 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.108563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.108656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.108800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.108882 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.109033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.109121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] standard_threads = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.109265 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.109344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.109503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.109582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.109663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.109745 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.109844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] use_journal = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.109976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.110056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] use_json = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.110186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] use_stderr = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.110272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.110356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] use_syslog = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.110431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.110610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:06.110692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.110869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.110967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.111111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.111198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.111347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.111427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.111579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.111668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.111822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.111915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.112064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.112158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.112306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.112388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.112542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.112628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.112781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.112869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.113008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.113090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.113249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.113339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.113483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.113565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.113715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.113797 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.113946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.114036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.114177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.114260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.114416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.114528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.114682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.114775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.114913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.114995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.115162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.115253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.115425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.115515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.115666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.115748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.115922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.116003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.116187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.116277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.116422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.116502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.116671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.116752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.116902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.116991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.117130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.117212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.117360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.117440 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.117594 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] resource.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.117747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.117827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.117905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.118072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.118156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.118304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.118392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.118562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.118646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.118806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.118886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.119039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.119134 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.119282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.119371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.119515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.119597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.119747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.119837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.119976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] role.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.120056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.120214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.120294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] role.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.120439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.120527 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.120678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.120757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.120913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.120994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.121150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.121240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.121392 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.121486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.121632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.121722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.121863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.121952 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.122102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.122182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.122333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.122420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.122586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.122720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.123366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.123448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.123611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.123693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.123881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.123965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.124125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.124206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.124364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.124532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.124624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.124767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.124849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.125013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.125102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.125258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.125349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.125502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.125584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.125739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.125819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.125964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.126054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.126201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.126282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.126447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.126591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.126704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.126800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.126934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.127017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.127163 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.127245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.127391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.127486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.127658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.127779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.127921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.128012 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.128131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.128221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.128367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.128450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.128604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.128684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.128833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.128926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.129067 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] token.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.129149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.129300 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.129382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.129542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.129630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.129775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.129857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.130017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.130097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.130258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.130348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.130521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.130608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.130761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.130843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.130997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.131088 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.131236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.131317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.131466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.131546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.131701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.131796 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.131936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.132018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.132178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.132306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.132423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.132552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.132635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.132851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.132933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.133092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.133173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.133392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.connection = **** {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.133482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.133633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.133716 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.133880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.134028 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.134161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.134300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.134430 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.134578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.134706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.134832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.134959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.135099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.135227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.135354 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.135479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.135605 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.135742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.135878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.136012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.136141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.136268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.136394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.136394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.136641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.136769 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.136897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.137023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.137156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] auth.external = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.137288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.137288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.137550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.137684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.137684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91981) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 05:39:06.137929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.137929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.138510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] auth.password = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.138768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] auth.token = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.139080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.139352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.139691 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.139957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.140298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.140579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.140887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.141157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] credential.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.141495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.141777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.142119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.142391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.142733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.143003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.143354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.143642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.143951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.144236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.144613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.144895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.145251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.145527 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.145880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.146157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] federation.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.146461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.146772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.147113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.147379 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.147719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.147990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.148337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.148607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.148954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.149224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.149542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.149813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.150258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.150549 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.150805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.151050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.151306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.151543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.151788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.152043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.152303 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.152542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.152780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.153017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.153268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.153521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.153774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.154060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.154313 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.154615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.154861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.155103 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.155342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.155585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.155823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.156078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.156324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.156569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.156823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.157061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.157313 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.157604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.157804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.158056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.158307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.158574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.158817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.159052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.159293 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.159532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.159779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.160019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.160258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.160511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.160767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.161001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.161237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.161486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.161736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.161976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.162214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.162453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.162709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.162954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.163215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.163451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.163700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.163950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.164450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.164450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.164658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.164897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.165137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.165375 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.165608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.165857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.166106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.166357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.166631 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.166881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.167114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.167362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.167601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.167849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.167939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.168088 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.168180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.168322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.168433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.168567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.168827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.169067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.169325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.170088 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.170330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.170598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.170845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.171083 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.171332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.171576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.171813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.172052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.172295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] resource.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.172550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.172800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.173038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.173291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.173544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.173782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.174031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.174281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.174552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.174793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] role.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.175042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] role.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.175276 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.175528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.175781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.176033 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.176283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.176535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.176783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.177033 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.177124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.177284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.177521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.177770 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.178021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.178272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.178543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.178965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.179204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.179457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.179709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.179958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.180212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.180450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.180710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.180965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.181207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.181443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.181678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.181915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.182170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.182429 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.182707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.182950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.183191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.183436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.183677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] token.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.183919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.184171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.184412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.184669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.184920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.185209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.185947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.186194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.186445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.186708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.186948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.187186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.187435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.187688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.187921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.188167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.188462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.188727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.connection = **** {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.188975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.189226 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.189468 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.189714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.189955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.190196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.190436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.190699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.190951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.191193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.191440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.191681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.191931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.192170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.192418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.192654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.192897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.193137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:06.193309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91980) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 05:39:06.204899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.205132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.219015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.219250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.219480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.227873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.246648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.246883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.286376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.344580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.380168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.380438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.424934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.425093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.447029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.447260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.447479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.447695 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.447910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.448123 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.467091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.467325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.467551 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.467772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.467986 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.468197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.473188 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.473423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.473651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.473872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.474089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.474304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.474609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.481194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.481431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.481660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.481887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.482673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.482896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.483111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.486043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.490818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.498569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.498800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.499023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.499242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.499457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.499667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.499878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91981) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.503870 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.504109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.504336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.504559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.504777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.504993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.505211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91980) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:06.517053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.517260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:39:06.517592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.517790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:39:06.517886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.518053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.518136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:39:06.518309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.518391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.518591 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.518689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:39:06.518844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.518926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.519106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.519365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.519861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.530342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.530643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.530911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.531191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.531308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.531587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.531854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.532134 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.539224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.540254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.542456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.542753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.543034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.543415 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.543687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.543783 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.543940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.544022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.544205 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.544287 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.544491 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.545067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.545274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.545355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.545542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.545623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.545802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.545886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.546075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.546346 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.546648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.546918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.571961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.573419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.573560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.573833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.575060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.575347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.579120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.579391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.579665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.580671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.580937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.581216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.586900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.587180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.588724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.589009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.592297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.592574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.592850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.593124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.593442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.593667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.593928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.594090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.594211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.594376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.594694 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.594990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.595283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.595569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.595849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.596116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.613140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.613415 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.615569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.615849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.617881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.620431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.621540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.621831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.624144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.624443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.626855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.629475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.630405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.633224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.634158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.634453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.634765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.635052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.635324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.635596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.637016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.637318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.637616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.637906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.639011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.639297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.655712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.656082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.656348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.656610 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.658917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.659203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.659471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.659741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.664452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.666138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.666411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.667689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.669395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.669479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.669732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.669899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.670179 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.670455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.670759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.672823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.673105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.673395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.673680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.674077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.684209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.684439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.684712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.684995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.685294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.685581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.685859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.687404 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.687689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.687974 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.688256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.688535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.688821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.689108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.705326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.708596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.709349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.711896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.712632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.713479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.715107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.715383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.715666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.715981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.718137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.719786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.720188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.720477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.723568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.725182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.727800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.728447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.728729 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.729006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.729283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.729418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.732733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.733029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.733316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.733593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.738825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.739111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.739387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.743184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.743488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.743785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.748122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91981) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.748398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.748673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.748974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.749242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.749551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.749838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.750110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.750390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.750676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.750941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.751197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.751455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.751713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91981) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.752170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91980) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:06.752482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.752765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.753060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.753345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.753625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.753913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.754194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.754503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.754800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.756230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.756544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.756826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.757148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91980) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:06.790678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5648864b99c0 pid: 91981 (default app) Jul 25 05:39:06.793990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb4aa9-d495-4085-91c6-e1c6a04d6ef4 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:06.794283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb4aa9-d495-4085-91c6-e1c6a04d6ef4 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:06.794626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb4aa9-d495-4085-91c6-e1c6a04d6ef4 None None] PATH_INFO: `/v3` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:06.796250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5648864b99c0 pid: 91980 (default app) Jul 25 05:39:06.798793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Mon Jul 25 05:39:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:15.386060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb7e8d-dbca-4697-9705-93764a05f42c None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:15.386623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb7e8d-dbca-4697-9705-93764a05f42c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:15.386719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb7e8d-dbca-4697-9705-93764a05f42c None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:15.391644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:15.396677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:15.397100 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:15.397223 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:15.465423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: INFO dbcounter [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] Registered counter for database keystone Jul 25 05:39:15.486260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG oslo_db.sqlalchemy.engines [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91981) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 05:39:15.492356 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writer thread running {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 05:39:15.667027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG passlib.handlers.bcrypt [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91981) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 05:39:15.667361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG passlib.handlers.bcrypt [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91981) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 05:39:15.721797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:15.723441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:15.843107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 05:39:16.009782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb7e8d-dbca-4697-9705-93764a05f42c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:16.081516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: INFO dbcounter [None req-aebb7e8d-dbca-4697-9705-93764a05f42c None None] Registered counter for database keystone Jul 25 05:39:16.100447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG oslo_db.sqlalchemy.engines [None req-aebb7e8d-dbca-4697-9705-93764a05f42c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91980) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 05:39:16.106585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writer thread running {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 05:39:16.229698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb7e8d-dbca-4697-9705-93764a05f42c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:16.234046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f38f3-b890-4e23-bce1-af55e96eafed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:16.234745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f38f3-b890-4e23-bce1-af55e96eafed admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:16.235029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f38f3-b890-4e23-bce1-af55e96eafed admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:16.235307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f38f3-b890-4e23-bce1-af55e96eafed admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:16.243403 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f38f3-b890-4e23-bce1-af55e96eafed admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:16.295020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WARNING keystone.common.rbac_enforcer.enforcer [None req-562f38f3-b890-4e23-bce1-af55e96eafed admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 05:39:16.295298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f38f3-b890-4e23-bce1-af55e96eafed admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:16.305160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Mon Jul 25 05:39:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:39:16.311636 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:16.326403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a9f2d-8901-4de2-a182-5dbf1044073d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:16.330846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589af408-0fbf-4057-953f-55e56bb6141b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:16.331969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589af408-0fbf-4057-953f-55e56bb6141b admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:16.332254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589af408-0fbf-4057-953f-55e56bb6141b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:16.332530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589af408-0fbf-4057-953f-55e56bb6141b admin admin] PATH_INFO: `/v3/projects/799d47101892496a8efa7ee2b0f5f24a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:16.334013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589af408-0fbf-4057-953f-55e56bb6141b admin admin] RBAC: Authorizing `identity:get_project(project_id=799d47101892496a8efa7ee2b0f5f24a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:16.386486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: WARNING keystone.common.rbac_enforcer.enforcer [None req-589af408-0fbf-4057-953f-55e56bb6141b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 05:39:16.386833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589af408-0fbf-4057-953f-55e56bb6141b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:16.391912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Mon Jul 25 05:39:16 2022] GET /identity/v3/projects/799d47101892496a8efa7ee2b0f5f24a => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:18.379764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5aee6e-50df-4f42-b1bd-7ccb3ab8b407 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.380270 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5aee6e-50df-4f42-b1bd-7ccb3ab8b407 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.380372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5aee6e-50df-4f42-b1bd-7ccb3ab8b407 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.384407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:18.390122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b9bfa-7d30-4a0f-b36b-c2009f3e83d7 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.390527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b9bfa-7d30-4a0f-b36b-c2009f3e83d7 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.390710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b9bfa-7d30-4a0f-b36b-c2009f3e83d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.409461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-ec7b9bfa-7d30-4a0f-b36b-c2009f3e83d7 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:18.411015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ec7b9bfa-7d30-4a0f-b36b-c2009f3e83d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:18.418867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 05:39:18.440945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5aee6e-50df-4f42-b1bd-7ccb3ab8b407 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:18.452815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5aee6e-50df-4f42-b1bd-7ccb3ab8b407 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.457006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235d1c0-6c78-4e13-b1d4-186d7fb1c5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:18.457631 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d1c0-6c78-4e13-b1d4-186d7fb1c5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.460449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d1c0-6c78-4e13-b1d4-186d7fb1c5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.460449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d1c0-6c78-4e13-b1d4-186d7fb1c5d1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.465730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235d1c0-6c78-4e13-b1d4-186d7fb1c5d1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:18.468071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235d1c0-6c78-4e13-b1d4-186d7fb1c5d1 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:18.476724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Mon Jul 25 05:39:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:39:18.482642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b9bfa-7d30-4a0f-b36b-c2009f3e83d7 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:18.491522 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b9bfa-7d30-4a0f-b36b-c2009f3e83d7 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.495926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9dea27-d4e0-4e31-84fa-9dc2ecaac033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:18.496778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dea27-d4e0-4e31-84fa-9dc2ecaac033 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.497061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dea27-d4e0-4e31-84fa-9dc2ecaac033 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.497336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dea27-d4e0-4e31-84fa-9dc2ecaac033 admin admin] PATH_INFO: `/v3/users/16be6f6629504bf5b97a9530841ecf24` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.498771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9dea27-d4e0-4e31-84fa-9dc2ecaac033 admin admin] RBAC: Authorizing `identity:get_user(user_id=16be6f6629504bf5b97a9530841ecf24)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:18.500588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9dea27-d4e0-4e31-84fa-9dc2ecaac033 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:18.502149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Mon Jul 25 05:39:18 2022] GET /identity/v3/users/16be6f6629504bf5b97a9530841ecf24 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:20.556727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59eaf-bfb3-4c72-90d2-f024d64f4d6f None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:20.557266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59eaf-bfb3-4c72-90d2-f024d64f4d6f None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:20.557266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59eaf-bfb3-4c72-90d2-f024d64f4d6f None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:20.558836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:20.564214 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f3dcc-9552-485e-9f33-823c3574fdab None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:20.564588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f3dcc-9552-485e-9f33-823c3574fdab None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:20.564876 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f3dcc-9552-485e-9f33-823c3574fdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:20.581694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-258f3dcc-9552-485e-9f33-823c3574fdab None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:20.582936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-258f3dcc-9552-485e-9f33-823c3574fdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:20.592142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d1fa5-7f49-4d22-819c-967504c907d6 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:20.592452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d1fa5-7f49-4d22-819c-967504c907d6 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:20.592735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d1fa5-7f49-4d22-819c-967504c907d6 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:20.594395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:20.599537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899996e-d6f0-4f9a-9369-c0af81614837 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:20.599896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899996e-d6f0-4f9a-9369-c0af81614837 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:20.600183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899996e-d6f0-4f9a-9369-c0af81614837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:20.640129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG passlib.handlers.bcrypt [None req-1899996e-d6f0-4f9a-9369-c0af81614837 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91980) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 05:39:20.640561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG passlib.handlers.bcrypt [None req-1899996e-d6f0-4f9a-9369-c0af81614837 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91980) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 05:39:20.663400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:20.685848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-1899996e-d6f0-4f9a-9369-c0af81614837 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:20.686126 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f3dcc-9552-485e-9f33-823c3574fdab None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:20.687467 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1899996e-d6f0-4f9a-9369-c0af81614837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:20.709349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:20.785711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:39:20.788583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f3dcc-9552-485e-9f33-823c3574fdab None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:20.807205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899996e-d6f0-4f9a-9369-c0af81614837 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:20.821556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54e06f-9cf3-4ba3-ab97-03708a71c061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:20.822355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54e06f-9cf3-4ba3-ab97-03708a71c061 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:20.822675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54e06f-9cf3-4ba3-ab97-03708a71c061 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:20.822981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54e06f-9cf3-4ba3-ab97-03708a71c061 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:20.834382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54e06f-9cf3-4ba3-ab97-03708a71c061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-7c54e06f-9cf3-4ba3-ab97-03708a71c061 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:20.845561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:20.846451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:20.847367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:20.852394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54e06f-9cf3-4ba3-ab97-03708a71c061 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:20.905976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899996e-d6f0-4f9a-9369-c0af81614837 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:20.908534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:20.935507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:20.943829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789103c-8fbc-4139-a807-04d50cca0fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:20.944818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789103c-8fbc-4139-a807-04d50cca0fa7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:20.945113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789103c-8fbc-4139-a807-04d50cca0fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:20.945395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789103c-8fbc-4139-a807-04d50cca0fa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:20.952906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54e06f-9cf3-4ba3-ab97-03708a71c061 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:20.986493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932a0fd-0223-4ce9-b689-b245ac61e041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:20.987173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932a0fd-0223-4ce9-b689-b245ac61e041 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:20.987456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932a0fd-0223-4ce9-b689-b245ac61e041 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:20.987733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932a0fd-0223-4ce9-b689-b245ac61e041 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:20.988484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932a0fd-0223-4ce9-b689-b245ac61e041 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:20.988788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932a0fd-0223-4ce9-b689-b245ac61e041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:20.990184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932a0fd-0223-4ce9-b689-b245ac61e041 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:21.002274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:21.007722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932a0fd-0223-4ce9-b689-b245ac61e041 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:21.013717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789103c-8fbc-4139-a807-04d50cca0fa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:21.015297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789103c-8fbc-4139-a807-04d50cca0fa7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:21.103913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932a0fd-0223-4ce9-b689-b245ac61e041 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:21.112058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:39:21.118324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789103c-8fbc-4139-a807-04d50cca0fa7 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:21.121001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:21.138754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da1f19-639f-4637-aafe-6680b6b71761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:21.139634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da1f19-639f-4637-aafe-6680b6b71761 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:21.139916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da1f19-639f-4637-aafe-6680b6b71761 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:21.140244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da1f19-639f-4637-aafe-6680b6b71761 None admin] PATH_INFO: `/v3/users/16be6f6629504bf5b97a9530841ecf24` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:21.141489 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da1f19-639f-4637-aafe-6680b6b71761 None admin] RBAC: Authorizing `identity:get_user(user_id=16be6f6629504bf5b97a9530841ecf24)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:21.142993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da1f19-639f-4637-aafe-6680b6b71761 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:21.144607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:21 2022] GET /identity/v3/users/16be6f6629504bf5b97a9530841ecf24 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:21.149846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da1f19-639f-4637-aafe-6680b6b71761 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:21.164030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:21.217207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789103c-8fbc-4139-a807-04d50cca0fa7 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:21.244908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da1f19-639f-4637-aafe-6680b6b71761 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:21.249925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a476b9-05fd-4f62-a5fc-8e0dd917a097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:21.250745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a476b9-05fd-4f62-a5fc-8e0dd917a097 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:21.251032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a476b9-05fd-4f62-a5fc-8e0dd917a097 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:21.251307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a476b9-05fd-4f62-a5fc-8e0dd917a097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:21.270100 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a476b9-05fd-4f62-a5fc-8e0dd917a097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:21.271794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a476b9-05fd-4f62-a5fc-8e0dd917a097 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:21.278191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3177ea-15f8-4690-a3ed-4fd6ed6d6048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:21.279561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3177ea-15f8-4690-a3ed-4fd6ed6d6048 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:21.279846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3177ea-15f8-4690-a3ed-4fd6ed6d6048 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:21.280124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3177ea-15f8-4690-a3ed-4fd6ed6d6048 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:21.281314 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3177ea-15f8-4690-a3ed-4fd6ed6d6048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:21.282820 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3177ea-15f8-4690-a3ed-4fd6ed6d6048 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:21.284420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:21.290017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3177ea-15f8-4690-a3ed-4fd6ed6d6048 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-92a476b9-05fd-4f62-a5fc-8e0dd917a097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:39:21.293115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:21.293919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:21.294813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:39:21.294813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:39:21.294813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:39:21.294813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:21.294813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:39:21.299924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a476b9-05fd-4f62-a5fc-8e0dd917a097 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:21.354758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:21.370176 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:21.389847 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3177ea-15f8-4690-a3ed-4fd6ed6d6048 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:21.403163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a476b9-05fd-4f62-a5fc-8e0dd917a097 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:21.423311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7d8c9-e030-4908-9dd7-0e734e9c73c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:21.423964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7d8c9-e030-4908-9dd7-0e734e9c73c5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:21.424243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7d8c9-e030-4908-9dd7-0e734e9c73c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:21.424520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7d8c9-e030-4908-9dd7-0e734e9c73c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:21.425358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7d8c9-e030-4908-9dd7-0e734e9c73c5 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.domain.id=default, user.id=16be6f6629504bf5b97a9530841ecf24) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:21.425673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7d8c9-e030-4908-9dd7-0e734e9c73c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:21.427142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7d8c9-e030-4908-9dd7-0e734e9c73c5 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:21.433710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 11/20] 10.222.0.17 () {56 vars in 1322 bytes} [Mon Jul 25 05:39:21 2022] GET /identity/v3/role_assignments?user.id=16be6f6629504bf5b97a9530841ecf24&scope.domain.id=default&role.id=7a833c45603a4830b888e58e1232712a => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:21.436553 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8f63d-2076-41bb-8620-aa84d415234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:21.437225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f63d-2076-41bb-8620-aa84d415234a None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:21.437512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f63d-2076-41bb-8620-aa84d415234a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:21.437786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f63d-2076-41bb-8620-aa84d415234a None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:21.438566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8f63d-2076-41bb-8620-aa84d415234a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:21.438874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8f63d-2076-41bb-8620-aa84d415234a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:21.440302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8f63d-2076-41bb-8620-aa84d415234a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:21.461296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 10/21] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:23.802125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0759a8e-bddf-4ef2-bad7-15d19ef64d56 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:23.803198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0759a8e-bddf-4ef2-bad7-15d19ef64d56 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:23.803679 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0759a8e-bddf-4ef2-bad7-15d19ef64d56 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:23.805282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 12/22] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:23.813966 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b6120-4479-42c7-8142-dc4f30483f5a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:23.814836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b6120-4479-42c7-8142-dc4f30483f5a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:23.815323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b6120-4479-42c7-8142-dc4f30483f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:23.837745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-fc2b6120-4479-42c7-8142-dc4f30483f5a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:23.839328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-fc2b6120-4479-42c7-8142-dc4f30483f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:23.932202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759867b-288f-4977-977c-7e7d074fee02 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:23.932819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759867b-288f-4977-977c-7e7d074fee02 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:23.933288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759867b-288f-4977-977c-7e7d074fee02 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:23.935479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 13/23] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:39:23.940959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f33eb-f56a-4d0f-b3af-fe1449280592 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:23.941592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f33eb-f56a-4d0f-b3af-fe1449280592 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:23.942382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f33eb-f56a-4d0f-b3af-fe1449280592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:23.950005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 11/24] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:39:23.979500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4c2f33eb-f56a-4d0f-b3af-fe1449280592 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:23.980494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b6120-4479-42c7-8142-dc4f30483f5a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:23.981101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4c2f33eb-f56a-4d0f-b3af-fe1449280592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:24.057688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:24.074893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 14/25] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:24.079936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2676f2-151d-467c-833d-36b6d6e59d61 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.079936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2676f2-151d-467c-833d-36b6d6e59d61 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.079936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2676f2-151d-467c-833d-36b6d6e59d61 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.079936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 15/26] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:24.084813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92fa69-d1af-4844-bc84-c62f3a516c30 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.085105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92fa69-d1af-4844-bc84-c62f3a516c30 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.085390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92fa69-d1af-4844-bc84-c62f3a516c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.103668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9a92fa69-d1af-4844-bc84-c62f3a516c30 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:24.104922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9a92fa69-d1af-4844-bc84-c62f3a516c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:24.174345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 16/27] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:39:24.178527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92fa69-d1af-4844-bc84-c62f3a516c30 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:24.191075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:24.208449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b6120-4479-42c7-8142-dc4f30483f5a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:24.292462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf2493-79d9-4649-9ae6-9d21bb51e222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:24.293346 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf2493-79d9-4649-9ae6-9d21bb51e222 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.293636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf2493-79d9-4649-9ae6-9d21bb51e222 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.293756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92fa69-d1af-4844-bc84-c62f3a516c30 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:24.293910 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf2493-79d9-4649-9ae6-9d21bb51e222 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.319668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf2493-79d9-4649-9ae6-9d21bb51e222 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:24.329968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2beb3-2cd7-41f9-9247-686f4ec5717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:24.330886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2beb3-2cd7-41f9-9247-686f4ec5717a None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.331338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2beb3-2cd7-41f9-9247-686f4ec5717a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.331770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2beb3-2cd7-41f9-9247-686f4ec5717a None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.332665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2beb3-2cd7-41f9-9247-686f4ec5717a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:24.334211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2beb3-2cd7-41f9-9247-686f4ec5717a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-73cf2493-79d9-4649-9ae6-9d21bb51e222 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:24.337060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:24.338157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 12/28] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:39:24.339142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf2493-79d9-4649-9ae6-9d21bb51e222 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:24.469079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf2493-79d9-4649-9ae6-9d21bb51e222 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:24.477199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:24.507713 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e7f13-660c-468b-9e84-526156703dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:24.508579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e7f13-660c-468b-9e84-526156703dff None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.508870 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e7f13-660c-468b-9e84-526156703dff None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.509151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e7f13-660c-468b-9e84-526156703dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.520328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e7f13-660c-468b-9e84-526156703dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:24.521817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e7f13-660c-468b-9e84-526156703dff None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-b85e7f13-660c-468b-9e84-526156703dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:39:24.541423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:24.542280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:24.543142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:39:24.543142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:39:24.543142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:39:24.543142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:24.543142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 13/29] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:24.543508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e7f13-660c-468b-9e84-526156703dff None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:24.573568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 17/30] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Jul 25 05:39:24 2022] POST /identity/v3/roles => generated 219 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:24.575468 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c819c8-bc90-4345-b85e-d2c679d61aa5 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.575954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c819c8-bc90-4345-b85e-d2c679d61aa5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.576371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c819c8-bc90-4345-b85e-d2c679d61aa5 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.578035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 18/31] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:24.578803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c819c8-bc90-4345-b85e-d2c679d61aa5 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:24.701902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:24.736956 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e7f13-660c-468b-9e84-526156703dff None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:24.774879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c819c8-bc90-4345-b85e-d2c679d61aa5 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:24.781275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d984f82-607b-48f0-8ca0-12d77c96429c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:24.781961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d984f82-607b-48f0-8ca0-12d77c96429c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.782249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d984f82-607b-48f0-8ca0-12d77c96429c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.782559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d984f82-607b-48f0-8ca0-12d77c96429c None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.783333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d984f82-607b-48f0-8ca0-12d77c96429c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:24.783647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d984f82-607b-48f0-8ca0-12d77c96429c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:24.785211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d984f82-607b-48f0-8ca0-12d77c96429c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:24.800129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 14/32] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:24.801232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb677e4-db00-49b1-a7c0-a00db734a87b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.801607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb677e4-db00-49b1-a7c0-a00db734a87b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.801799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb677e4-db00-49b1-a7c0-a00db734a87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.828401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ffb677e4-db00-49b1-a7c0-a00db734a87b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:24.829545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ffb677e4-db00-49b1-a7c0-a00db734a87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:24.829790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfba6c-313e-4fe8-9fda-aab1dd3f7b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:24.830496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfba6c-313e-4fe8-9fda-aab1dd3f7b29 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.830843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfba6c-313e-4fe8-9fda-aab1dd3f7b29 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.831129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfba6c-313e-4fe8-9fda-aab1dd3f7b29 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.832218 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfba6c-313e-4fe8-9fda-aab1dd3f7b29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:24.833473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfba6c-313e-4fe8-9fda-aab1dd3f7b29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:24.835074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfba6c-313e-4fe8-9fda-aab1dd3f7b29 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:24.850887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:24.858011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 19/33] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:24.859029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfba6c-313e-4fe8-9fda-aab1dd3f7b29 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:24.925773 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 15/34] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:24.927002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb677e4-db00-49b1-a7c0-a00db734a87b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:24.986927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfba6c-313e-4fe8-9fda-aab1dd3f7b29 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.020072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:25.034936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de00bb-7c4f-4de1-9be3-8ec6f3337127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.036596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de00bb-7c4f-4de1-9be3-8ec6f3337127 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.037036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de00bb-7c4f-4de1-9be3-8ec6f3337127 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.038025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de00bb-7c4f-4de1-9be3-8ec6f3337127 None admin] PATH_INFO: `/v3/users/16be6f6629504bf5b97a9530841ecf24` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.039700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de00bb-7c4f-4de1-9be3-8ec6f3337127 None admin] RBAC: Authorizing `identity:get_user(user_id=16be6f6629504bf5b97a9530841ecf24)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.042076 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de00bb-7c4f-4de1-9be3-8ec6f3337127 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.045853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 20/35] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:24 2022] GET /identity/v3/users/16be6f6629504bf5b97a9530841ecf24 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:25.047132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de00bb-7c4f-4de1-9be3-8ec6f3337127 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:25.056058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb677e4-db00-49b1-a7c0-a00db734a87b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.088460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7d691-a504-4db3-b0ea-b97ce64857f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.089451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7d691-a504-4db3-b0ea-b97ce64857f9 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.089926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7d691-a504-4db3-b0ea-b97ce64857f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.090403 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7d691-a504-4db3-b0ea-b97ce64857f9 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.092181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7d691-a504-4db3-b0ea-b97ce64857f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.093888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7d691-a504-4db3-b0ea-b97ce64857f9 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.263837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de00bb-7c4f-4de1-9be3-8ec6f3337127 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.299921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d9fde-b349-468e-a244-90aa93eb968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.300596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d9fde-b349-468e-a244-90aa93eb968b None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.300880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d9fde-b349-468e-a244-90aa93eb968b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.301162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d9fde-b349-468e-a244-90aa93eb968b None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.301927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d9fde-b349-468e-a244-90aa93eb968b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.303352 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d9fde-b349-468e-a244-90aa93eb968b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.319126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 16/36] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:39:24 2022] POST /identity/v3/projects => generated 301 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:25.320280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7d691-a504-4db3-b0ea-b97ce64857f9 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:25.323362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:25.378520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 21/37] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Jul 25 05:39:25 2022] POST /identity/v3/roles => generated 225 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:25.378520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca3a37-ec29-41c2-b9d0-e22dab0dc375 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.378520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca3a37-ec29-41c2-b9d0-e22dab0dc375 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.378520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca3a37-ec29-41c2-b9d0-e22dab0dc375 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.378520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 22/38] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:25.378520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56dd8-ce24-49ec-81ea-e584bbcc9b0f None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.378520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56dd8-ce24-49ec-81ea-e584bbcc9b0f None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.379089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56dd8-ce24-49ec-81ea-e584bbcc9b0f None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.380027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 23/39] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:25.385479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fe37e-aa1c-4ab0-a9e5-a2466fa1913b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.385787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fe37e-aa1c-4ab0-a9e5-a2466fa1913b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.386054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fe37e-aa1c-4ab0-a9e5-a2466fa1913b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.420774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-3c6fe37e-aa1c-4ab0-a9e5-a2466fa1913b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:25.423855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3c6fe37e-aa1c-4ab0-a9e5-a2466fa1913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:25.461304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7d691-a504-4db3-b0ea-b97ce64857f9 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.497890 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:25.508267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd1503-ac16-4d01-8364-b2c33d733983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.509491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd1503-ac16-4d01-8364-b2c33d733983 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.510004 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd1503-ac16-4d01-8364-b2c33d733983 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.510455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd1503-ac16-4d01-8364-b2c33d733983 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.514125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd1503-ac16-4d01-8364-b2c33d733983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.516212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd1503-ac16-4d01-8364-b2c33d733983 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.517757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 17/40] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:25.522224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eea2cc-1f74-4253-a3c4-f82633d29a6e None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.522224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eea2cc-1f74-4253-a3c4-f82633d29a6e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.522224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eea2cc-1f74-4253-a3c4-f82633d29a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.534233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 24/41] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:39:25.537214 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fe37e-aa1c-4ab0-a9e5-a2466fa1913b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:25.542261 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-63eea2cc-1f74-4253-a3c4-f82633d29a6e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:25.543898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-63eea2cc-1f74-4253-a3c4-f82633d29a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:25.624544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:25.628725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 18/42] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:25.629401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eea2cc-1f74-4253-a3c4-f82633d29a6e None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:25.661353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fe37e-aa1c-4ab0-a9e5-a2466fa1913b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.695178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7229a-b4e2-4652-a596-09fcbc26ebf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.696119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7229a-b4e2-4652-a596-09fcbc26ebf4 None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.696430 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7229a-b4e2-4652-a596-09fcbc26ebf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.696715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7229a-b4e2-4652-a596-09fcbc26ebf4 None admin] PATH_INFO: `/v3/domains/default/users/16be6f6629504bf5b97a9530841ecf24/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.698776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7229a-b4e2-4652-a596-09fcbc26ebf4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=16be6f6629504bf5b97a9530841ecf24, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.700770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7229a-b4e2-4652-a596-09fcbc26ebf4 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.728868 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eea2cc-1f74-4253-a3c4-f82633d29a6e None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.761564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae17e69-dc2c-4d7b-aa18-9568ef1b1dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.762366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae17e69-dc2c-4d7b-aa18-9568ef1b1dda None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.762690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae17e69-dc2c-4d7b-aa18-9568ef1b1dda None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.763726 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae17e69-dc2c-4d7b-aa18-9568ef1b1dda None admin] PATH_INFO: `/v3/domains/default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.764968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae17e69-dc2c-4d7b-aa18-9568ef1b1dda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.766425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae17e69-dc2c-4d7b-aa18-9568ef1b1dda None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.768087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 19/43] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:25.768840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae17e69-dc2c-4d7b-aa18-9568ef1b1dda None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:25.783493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 25/44] 10.222.0.17 () {58 vars in 1361 bytes} [Mon Jul 25 05:39:25 2022] PUT /identity/v3/domains/default/users/16be6f6629504bf5b97a9530841ecf24/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:25.784301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7229a-b4e2-4652-a596-09fcbc26ebf4 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:25.839131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:25.852793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:25.890246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae17e69-dc2c-4d7b-aa18-9568ef1b1dda None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.908810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7229a-b4e2-4652-a596-09fcbc26ebf4 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.932326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e8276-e65e-4bce-8100-5aadc2286f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.933149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e8276-e65e-4bce-8100-5aadc2286f02 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.933350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e8276-e65e-4bce-8100-5aadc2286f02 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.934214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e8276-e65e-4bce-8100-5aadc2286f02 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.934335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e8276-e65e-4bce-8100-5aadc2286f02 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.937628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e8276-e65e-4bce-8100-5aadc2286f02 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.953289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bc44e-7a02-4e0e-b53e-fcbf53554fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.953963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bc44e-7a02-4e0e-b53e-fcbf53554fd7 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.954250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bc44e-7a02-4e0e-b53e-fcbf53554fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.954578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bc44e-7a02-4e0e-b53e-fcbf53554fd7 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.955397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bc44e-7a02-4e0e-b53e-fcbf53554fd7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.956774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bc44e-7a02-4e0e-b53e-fcbf53554fd7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:26.009742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 20/45] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Jul 25 05:39:25 2022] POST /identity/v3/roles => generated 223 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:26.040005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 26/46] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:39:25 2022] POST /identity/v3/projects => generated 312 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:27.803535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166ceb9-a67c-4e8e-ac3c-c75327542e63 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:27.804049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166ceb9-a67c-4e8e-ac3c-c75327542e63 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:27.804154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166ceb9-a67c-4e8e-ac3c-c75327542e63 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:27.805474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 21/47] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:27.810788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853851c-f67b-4e43-9dec-dd873ec98e1b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:27.811185 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853851c-f67b-4e43-9dec-dd873ec98e1b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:27.811359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853851c-f67b-4e43-9dec-dd873ec98e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:27.828413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4853851c-f67b-4e43-9dec-dd873ec98e1b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:27.829608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4853851c-f67b-4e43-9dec-dd873ec98e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:27.898493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 27/48] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:27.921382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166ceb9-a67c-4e8e-ac3c-c75327542e63 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:28.024511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166ceb9-a67c-4e8e-ac3c-c75327542e63 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:28.050161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:28.057540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a11d1-d46b-496d-afcd-7e65b77f28ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:28.058274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a11d1-d46b-496d-afcd-7e65b77f28ac None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:28.058598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a11d1-d46b-496d-afcd-7e65b77f28ac None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:28.058883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a11d1-d46b-496d-afcd-7e65b77f28ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:28.069779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a11d1-d46b-496d-afcd-7e65b77f28ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-2e3a11d1-d46b-496d-afcd-7e65b77f28ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:28.079441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:28.080322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:28.081087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 22/49] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:28.086824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853851c-f67b-4e43-9dec-dd873ec98e1b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:28.116127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:28.134737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65626401-666a-4b0e-b333-4fe0a4ed3b5c None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:28.135022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65626401-666a-4b0e-b333-4fe0a4ed3b5c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:28.135304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65626401-666a-4b0e-b333-4fe0a4ed3b5c None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:28.136683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 23/50] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:39:28.142509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238aa14-3164-439e-8b93-0eb3bd5520c3 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:28.142788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238aa14-3164-439e-8b93-0eb3bd5520c3 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:28.143069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238aa14-3164-439e-8b93-0eb3bd5520c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:28.159212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c238aa14-3164-439e-8b93-0eb3bd5520c3 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:28.160400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c238aa14-3164-439e-8b93-0eb3bd5520c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:28.195091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853851c-f67b-4e43-9dec-dd873ec98e1b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:28.225225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 24/51] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:39:28.229969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a8173-653f-4b35-bdf9-434b316936e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:28.230678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a8173-653f-4b35-bdf9-434b316936e2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:28.230969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a8173-653f-4b35-bdf9-434b316936e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:28.231259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a8173-653f-4b35-bdf9-434b316936e2 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:28.232052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a8173-653f-4b35-bdf9-434b316936e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:28.232365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a8173-653f-4b35-bdf9-434b316936e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:28.233766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a8173-653f-4b35-bdf9-434b316936e2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:28.241839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 28/52] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:28.247958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238aa14-3164-439e-8b93-0eb3bd5520c3 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:28.248078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a8173-653f-4b35-bdf9-434b316936e2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:28.330296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:28.345192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a8173-653f-4b35-bdf9-434b316936e2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:28.346187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238aa14-3164-439e-8b93-0eb3bd5520c3 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:28.380847 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862fef20-293c-4038-9d39-b6ab5960f970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:28.381676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fef20-293c-4038-9d39-b6ab5960f970 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:28.381966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fef20-293c-4038-9d39-b6ab5960f970 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:28.382252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862fef20-293c-4038-9d39-b6ab5960f970 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:28.383560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862fef20-293c-4038-9d39-b6ab5960f970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:28.385054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862fef20-293c-4038-9d39-b6ab5960f970 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:28.386806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 29/53] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:28.393431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862fef20-293c-4038-9d39-b6ab5960f970 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:28.469303 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:28.486370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6450f-202c-4240-be9d-cea4d2d09f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:28.487286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6450f-202c-4240-be9d-cea4d2d09f20 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:28.487568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6450f-202c-4240-be9d-cea4d2d09f20 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:28.487846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6450f-202c-4240-be9d-cea4d2d09f20 None admin] PATH_INFO: `/v3/users/16be6f6629504bf5b97a9530841ecf24` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:28.489300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f6450f-202c-4240-be9d-cea4d2d09f20 None admin] RBAC: Authorizing `identity:get_user(user_id=16be6f6629504bf5b97a9530841ecf24)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:28.490849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f6450f-202c-4240-be9d-cea4d2d09f20 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:28.492478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 25/54] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:28 2022] GET /identity/v3/users/16be6f6629504bf5b97a9530841ecf24 => generated 247 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:28.498021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6450f-202c-4240-be9d-cea4d2d09f20 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:28.499760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862fef20-293c-4038-9d39-b6ab5960f970 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:28.534933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047995b-5b1e-464e-ad68-3bdf687d28e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:28.535589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047995b-5b1e-464e-ad68-3bdf687d28e7 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:28.535873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047995b-5b1e-464e-ad68-3bdf687d28e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:28.536152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047995b-5b1e-464e-ad68-3bdf687d28e7 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:28.536864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047995b-5b1e-464e-ad68-3bdf687d28e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:28.538313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047995b-5b1e-464e-ad68-3bdf687d28e7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:28.597763 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6450f-202c-4240-be9d-cea4d2d09f20 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:28.630697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef3417-0978-4642-9925-e57b9fb317ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:28.631513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef3417-0978-4642-9925-e57b9fb317ed None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:28.631795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef3417-0978-4642-9925-e57b9fb317ed None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:28.632070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef3417-0978-4642-9925-e57b9fb317ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:28.633336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef3417-0978-4642-9925-e57b9fb317ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:28.637436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef3417-0978-4642-9925-e57b9fb317ed None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:28.639926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 26/55] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:28.645655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef3417-0978-4642-9925-e57b9fb317ed None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:28.675462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 30/56] 10.222.0.17 () {60 vars in 1140 bytes} [Mon Jul 25 05:39:28 2022] POST /identity/v3/projects => generated 298 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:39:28.702455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:28.750758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef3417-0978-4642-9925-e57b9fb317ed None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:28.791333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea400c4-cc73-47db-bbef-8ab32a0ccc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:28.792085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea400c4-cc73-47db-bbef-8ab32a0ccc78 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:28.792375 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea400c4-cc73-47db-bbef-8ab32a0ccc78 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:28.792682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea400c4-cc73-47db-bbef-8ab32a0ccc78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:28.793775 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea400c4-cc73-47db-bbef-8ab32a0ccc78 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.domain.id=default, user.id=16be6f6629504bf5b97a9530841ecf24) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:28.794089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea400c4-cc73-47db-bbef-8ab32a0ccc78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:28.795607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea400c4-cc73-47db-bbef-8ab32a0ccc78 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:28.805441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 27/57] 10.222.0.17 () {56 vars in 1322 bytes} [Mon Jul 25 05:39:28 2022] GET /identity/v3/role_assignments?user.id=16be6f6629504bf5b97a9530841ecf24&scope.domain.id=default&role.id=7a833c45603a4830b888e58e1232712a => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:30.711661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291fda1-a4e5-45cd-a898-63900190f629 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:30.712173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291fda1-a4e5-45cd-a898-63900190f629 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:30.712257 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291fda1-a4e5-45cd-a898-63900190f629 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:30.713622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 31/58] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:30.719455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f6b76-bb9b-4655-b18b-c73f4b18ba0b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:30.719822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f6b76-bb9b-4655-b18b-c73f4b18ba0b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:30.720030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f6b76-bb9b-4655-b18b-c73f4b18ba0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:30.736484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-938f6b76-bb9b-4655-b18b-c73f4b18ba0b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:30.737686 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-938f6b76-bb9b-4655-b18b-c73f4b18ba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:30.811242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 28/59] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:30.835995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291fda1-a4e5-45cd-a898-63900190f629 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:30.845873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:30.936576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291fda1-a4e5-45cd-a898-63900190f629 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:30.970091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4d696-7ac5-4034-ab30-54d78203f062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:30.970848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4d696-7ac5-4034-ab30-54d78203f062 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:30.971132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4d696-7ac5-4034-ab30-54d78203f062 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:30.971406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4d696-7ac5-4034-ab30-54d78203f062 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:30.972566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4d696-7ac5-4034-ab30-54d78203f062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:30.974019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4d696-7ac5-4034-ab30-54d78203f062 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:30.975655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 32/60] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:30.981129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f6b76-bb9b-4655-b18b-c73f4b18ba0b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:31.027844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:31.080543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f6b76-bb9b-4655-b18b-c73f4b18ba0b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:31.111989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d21906-a2ce-4cf5-8377-5df8ccada659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:31.112651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d21906-a2ce-4cf5-8377-5df8ccada659 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:31.112932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d21906-a2ce-4cf5-8377-5df8ccada659 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:31.113209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d21906-a2ce-4cf5-8377-5df8ccada659 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:31.113918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d21906-a2ce-4cf5-8377-5df8ccada659 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:31.115389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d21906-a2ce-4cf5-8377-5df8ccada659 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:31.210737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 29/61] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:39:30 2022] POST /identity/v3/users => generated 275 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:34.384235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b146b9-8986-4bc7-9620-c8b396b53c5c None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.384781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b146b9-8986-4bc7-9620-c8b396b53c5c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.384781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b146b9-8986-4bc7-9620-c8b396b53c5c None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.386177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 33/62] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:34.400894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289f05e-1307-4564-99f2-dc5e9c0c09cd None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.401371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289f05e-1307-4564-99f2-dc5e9c0c09cd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.409693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289f05e-1307-4564-99f2-dc5e9c0c09cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.423802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-9289f05e-1307-4564-99f2-dc5e9c0c09cd None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:34.425161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9289f05e-1307-4564-99f2-dc5e9c0c09cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:34.520767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 30/63] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:34.545278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72911c42-823e-4a5d-a9fb-f3fdd9fbc51e None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.545895 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72911c42-823e-4a5d-a9fb-f3fdd9fbc51e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.546369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72911c42-823e-4a5d-a9fb-f3fdd9fbc51e None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.547981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 34/64] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:34.553164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25939e19-5f39-436a-b052-b0cf6b7bb663 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.553687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25939e19-5f39-436a-b052-b0cf6b7bb663 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.554158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25939e19-5f39-436a-b052-b0cf6b7bb663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.556022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72911c42-823e-4a5d-a9fb-f3fdd9fbc51e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:34.581688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-25939e19-5f39-436a-b052-b0cf6b7bb663 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:34.583241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-25939e19-5f39-436a-b052-b0cf6b7bb663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:34.612345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:34.652925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:34.680227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 31/65] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:34.681887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c923b-0145-4c78-9dab-1756a81cd18c None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.682183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c923b-0145-4c78-9dab-1756a81cd18c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.682462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c923b-0145-4c78-9dab-1756a81cd18c None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.683770 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 32/66] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:34.691826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5fa38-615d-4352-9201-b604807c1a24 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.700240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5fa38-615d-4352-9201-b604807c1a24 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.700240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5fa38-615d-4352-9201-b604807c1a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.700406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72911c42-823e-4a5d-a9fb-f3fdd9fbc51e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:34.710939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-31b5fa38-615d-4352-9201-b604807c1a24 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:34.712156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-31b5fa38-615d-4352-9201-b604807c1a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:34.742865 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d5998-6088-46d9-9b5c-7754de78a2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:34.743901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d5998-6088-46d9-9b5c-7754de78a2bf None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.744326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d5998-6088-46d9-9b5c-7754de78a2bf None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.744741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d5998-6088-46d9-9b5c-7754de78a2bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.756312 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d5998-6088-46d9-9b5c-7754de78a2bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-b09d5998-6088-46d9-9b5c-7754de78a2bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:34.764988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:34.765871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:34.766688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 35/67] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:39:34.767343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d5998-6088-46d9-9b5c-7754de78a2bf None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:34.794671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 33/68] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:34.804980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65873d8b-3562-47c1-9faa-690150730609 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.805500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65873d8b-3562-47c1-9faa-690150730609 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.806000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65873d8b-3562-47c1-9faa-690150730609 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.807588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 34/69] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:34.808493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65873d8b-3562-47c1-9faa-690150730609 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:34.880409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d5998-6088-46d9-9b5c-7754de78a2bf None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:34.917247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10c9cd-11eb-4ac2-b868-876ff81dc90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:34.918246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c9cd-11eb-4ac2-b868-876ff81dc90c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.918361 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:34.918561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c9cd-11eb-4ac2-b868-876ff81dc90c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.918861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10c9cd-11eb-4ac2-b868-876ff81dc90c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.920928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:34.926272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65873d8b-3562-47c1-9faa-690150730609 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:34.930386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10c9cd-11eb-4ac2-b868-876ff81dc90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-af10c9cd-11eb-4ac2-b868-876ff81dc90c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:34.938861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:34.948310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 36/70] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:39:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:34.949277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b6fac-b316-4ca0-b75a-32efb4da8a54 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.950858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b6fac-b316-4ca0-b75a-32efb4da8a54 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.950858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b6fac-b316-4ca0-b75a-32efb4da8a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.962029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599f94c-2890-48bd-bbbf-bf1fc9e0ffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:34.962410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9e3b6fac-b316-4ca0-b75a-32efb4da8a54 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:34.962759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599f94c-2890-48bd-bbbf-bf1fc9e0ffb1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:34.963040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599f94c-2890-48bd-bbbf-bf1fc9e0ffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:34.963338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599f94c-2890-48bd-bbbf-bf1fc9e0ffb1 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:34.963652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9e3b6fac-b316-4ca0-b75a-32efb4da8a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:34.964150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599f94c-2890-48bd-bbbf-bf1fc9e0ffb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:34.964470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599f94c-2890-48bd-bbbf-bf1fc9e0ffb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:34.967551 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599f94c-2890-48bd-bbbf-bf1fc9e0ffb1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:34.980409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 35/71] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:34.980409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599f94c-2890-48bd-bbbf-bf1fc9e0ffb1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.038458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 37/72] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:35.039419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b6fac-b316-4ca0-b75a-32efb4da8a54 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.089454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599f94c-2890-48bd-bbbf-bf1fc9e0ffb1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.121754 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36017e11-b0da-47bd-847d-e54ea25e0f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.123182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36017e11-b0da-47bd-847d-e54ea25e0f52 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.123474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36017e11-b0da-47bd-847d-e54ea25e0f52 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.123751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36017e11-b0da-47bd-847d-e54ea25e0f52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.134285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36017e11-b0da-47bd-847d-e54ea25e0f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-36017e11-b0da-47bd-847d-e54ea25e0f52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:35.143244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:35.144137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:35.144920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:35.145521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b6fac-b316-4ca0-b75a-32efb4da8a54 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.145591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 36/73] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:39:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:35.145591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36017e11-b0da-47bd-847d-e54ea25e0f52 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.163714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:35.167717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:35.178881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4d600-9c00-446c-acf7-9a2559598dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.179558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4d600-9c00-446c-acf7-9a2559598dc4 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.179846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4d600-9c00-446c-acf7-9a2559598dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.180130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4d600-9c00-446c-acf7-9a2559598dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.180888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4d600-9c00-446c-acf7-9a2559598dc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:35.187467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4d600-9c00-446c-acf7-9a2559598dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:35.187467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4d600-9c00-446c-acf7-9a2559598dc4 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:35.191352 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 38/74] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:35.192659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5dd55-41d4-4111-a480-121187a5b8fc None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.193038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5dd55-41d4-4111-a480-121187a5b8fc None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.193320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5dd55-41d4-4111-a480-121187a5b8fc None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.194776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 39/75] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:35.196773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5dd55-41d4-4111-a480-121187a5b8fc None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.245251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36017e11-b0da-47bd-847d-e54ea25e0f52 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.279639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e0da1-392f-4f53-9c1d-dc905f55f1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.280518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e0da1-392f-4f53-9c1d-dc905f55f1b7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e0da1-392f-4f53-9c1d-dc905f55f1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.281071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e0da1-392f-4f53-9c1d-dc905f55f1b7 None admin] PATH_INFO: `/v3/users/73a2be1f4ead4bf0a051368674237133` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.296289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e0da1-392f-4f53-9c1d-dc905f55f1b7 None admin] RBAC: Authorizing `identity:get_user(user_id=73a2be1f4ead4bf0a051368674237133)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:35.296966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5dd55-41d4-4111-a480-121187a5b8fc None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.297739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e0da1-392f-4f53-9c1d-dc905f55f1b7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:35.299626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 37/76] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/users/73a2be1f4ead4bf0a051368674237133 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:35.300202 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e0da1-392f-4f53-9c1d-dc905f55f1b7 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.330884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d1eb5-952f-4dbd-80ee-bc80554fb344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.331673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d1eb5-952f-4dbd-80ee-bc80554fb344 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.331982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d1eb5-952f-4dbd-80ee-bc80554fb344 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.332266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d1eb5-952f-4dbd-80ee-bc80554fb344 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.342543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d1eb5-952f-4dbd-80ee-bc80554fb344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-f38d1eb5-952f-4dbd-80ee-bc80554fb344 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:35.350327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:35.351397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:35.352159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 40/77] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:35.352788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d1eb5-952f-4dbd-80ee-bc80554fb344 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.378409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:35.401809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:35.404075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e0da1-392f-4f53-9c1d-dc905f55f1b7 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.437491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092a8c3-3393-4716-a13d-55475ed9c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.438095 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092a8c3-3393-4716-a13d-55475ed9c992 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.438380 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092a8c3-3393-4716-a13d-55475ed9c992 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.438685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092a8c3-3393-4716-a13d-55475ed9c992 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.439495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092a8c3-3393-4716-a13d-55475ed9c992 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:35.440925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092a8c3-3393-4716-a13d-55475ed9c992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:35.442347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092a8c3-3393-4716-a13d-55475ed9c992 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:35.452832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 38/78] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:35.452832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37e8c9-b239-40a2-8199-8141ccf6bcd2 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.452832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37e8c9-b239-40a2-8199-8141ccf6bcd2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.452832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37e8c9-b239-40a2-8199-8141ccf6bcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.458943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d1eb5-952f-4dbd-80ee-bc80554fb344 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.475578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-9d37e8c9-b239-40a2-8199-8141ccf6bcd2 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:35.476803 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9d37e8c9-b239-40a2-8199-8141ccf6bcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:35.497335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d51da5-3d55-4fad-ac29-0ccedabd88ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.498220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d51da5-3d55-4fad-ac29-0ccedabd88ab None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.498535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d51da5-3d55-4fad-ac29-0ccedabd88ab None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.498820 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d51da5-3d55-4fad-ac29-0ccedabd88ab None admin] PATH_INFO: `/v3/users/16be6f6629504bf5b97a9530841ecf24` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.500037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d51da5-3d55-4fad-ac29-0ccedabd88ab None admin] RBAC: Authorizing `identity:get_user(user_id=16be6f6629504bf5b97a9530841ecf24)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:35.501500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d51da5-3d55-4fad-ac29-0ccedabd88ab None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:35.503168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 41/79] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/users/16be6f6629504bf5b97a9530841ecf24 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:35.507264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d51da5-3d55-4fad-ac29-0ccedabd88ab None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.542649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 39/80] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:35.547735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37e8c9-b239-40a2-8199-8141ccf6bcd2 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.610697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d51da5-3d55-4fad-ac29-0ccedabd88ab None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.611805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:35.624655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:35.644414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df75b0-d586-4f4c-9906-7c66aadf2475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.645062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df75b0-d586-4f4c-9906-7c66aadf2475 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.645345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df75b0-d586-4f4c-9906-7c66aadf2475 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.645435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37e8c9-b239-40a2-8199-8141ccf6bcd2 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.645620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df75b0-d586-4f4c-9906-7c66aadf2475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.678516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df02207-43ad-4719-b04b-16a9624cd0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.679199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02207-43ad-4719-b04b-16a9624cd0ba None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.679487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02207-43ad-4719-b04b-16a9624cd0ba None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.679780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02207-43ad-4719-b04b-16a9624cd0ba None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.680530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df02207-43ad-4719-b04b-16a9624cd0ba None admin] RBAC: Adding query filter params (name=member) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:35.680844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df02207-43ad-4719-b04b-16a9624cd0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:35.682245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df02207-43ad-4719-b04b-16a9624cd0ba None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:35.695362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 40/81] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:35.695362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df02207-43ad-4719-b04b-16a9624cd0ba None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.716438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df75b0-d586-4f4c-9906-7c66aadf2475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:35.717921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df75b0-d586-4f4c-9906-7c66aadf2475 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:35.796344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df02207-43ad-4719-b04b-16a9624cd0ba None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.813768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 42/82] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:35.814781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df75b0-d586-4f4c-9906-7c66aadf2475 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.821907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:35.830023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:35.830171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32348f7-2b24-468a-bc5b-dea32226126c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.830966 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32348f7-2b24-468a-bc5b-dea32226126c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.831254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32348f7-2b24-468a-bc5b-dea32226126c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.831608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32348f7-2b24-468a-bc5b-dea32226126c None admin] PATH_INFO: `/v3/users/73a2be1f4ead4bf0a051368674237133` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.832843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32348f7-2b24-468a-bc5b-dea32226126c None admin] RBAC: Authorizing `identity:get_user(user_id=73a2be1f4ead4bf0a051368674237133)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:35.834299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32348f7-2b24-468a-bc5b-dea32226126c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:35.836023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 41/83] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/users/73a2be1f4ead4bf0a051368674237133 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:35.836781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32348f7-2b24-468a-bc5b-dea32226126c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:35.924443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df75b0-d586-4f4c-9906-7c66aadf2475 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.937876 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32348f7-2b24-468a-bc5b-dea32226126c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:35.958520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814dedf-4a38-43a0-b049-5053d6e0c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.959137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814dedf-4a38-43a0-b049-5053d6e0c750 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.959421 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814dedf-4a38-43a0-b049-5053d6e0c750 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.959701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814dedf-4a38-43a0-b049-5053d6e0c750 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.970559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5282680-bc41-4284-a55c-9e913f3d8993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:35.971360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5282680-bc41-4284-a55c-9e913f3d8993 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:35.971637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5282680-bc41-4284-a55c-9e913f3d8993 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:35.971910 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5282680-bc41-4284-a55c-9e913f3d8993 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:35.973068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5282680-bc41-4284-a55c-9e913f3d8993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:35.974573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5282680-bc41-4284-a55c-9e913f3d8993 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:35.976370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 42/84] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:35.979355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5282680-bc41-4284-a55c-9e913f3d8993 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.030309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814dedf-4a38-43a0-b049-5053d6e0c750 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:36.031851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814dedf-4a38-43a0-b049-5053d6e0c750 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:36.041569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:36.046233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:36.081292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5282680-bc41-4284-a55c-9e913f3d8993 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:36.116544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef91fed-e744-44d1-9e88-310e22d28c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:36.117432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef91fed-e744-44d1-9e88-310e22d28c3a None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:36.117717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef91fed-e744-44d1-9e88-310e22d28c3a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:36.117992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef91fed-e744-44d1-9e88-310e22d28c3a None admin] PATH_INFO: `/v3/users/73a2be1f4ead4bf0a051368674237133` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:36.119225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef91fed-e744-44d1-9e88-310e22d28c3a None admin] RBAC: Authorizing `identity:get_user(user_id=73a2be1f4ead4bf0a051368674237133)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:36.120683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef91fed-e744-44d1-9e88-310e22d28c3a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:36.122992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 43/85] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/users/73a2be1f4ead4bf0a051368674237133 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:36.126687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef91fed-e744-44d1-9e88-310e22d28c3a None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.130505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 43/86] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:36.131310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814dedf-4a38-43a0-b049-5053d6e0c750 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.233074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef91fed-e744-44d1-9e88-310e22d28c3a None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:36.235856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814dedf-4a38-43a0-b049-5053d6e0c750 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:36.250754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:36.262631 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:36.267973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618c617-8981-4dd0-96a7-841c1c4a6141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:36.268944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618c617-8981-4dd0-96a7-841c1c4a6141 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:36.269302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618c617-8981-4dd0-96a7-841c1c4a6141 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:36.269676 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618c617-8981-4dd0-96a7-841c1c4a6141 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:36.270964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618c617-8981-4dd0-96a7-841c1c4a6141 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:36.272418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618c617-8981-4dd0-96a7-841c1c4a6141 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:36.272973 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f85bd2-66fd-4033-96f9-9dc0053fc90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:36.273632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f85bd2-66fd-4033-96f9-9dc0053fc90d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:36.273919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f85bd2-66fd-4033-96f9-9dc0053fc90d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:36.274132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 44/87] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:36 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:36.274215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f85bd2-66fd-4033-96f9-9dc0053fc90d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:36.274906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618c617-8981-4dd0-96a7-841c1c4a6141 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.346608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f85bd2-66fd-4033-96f9-9dc0053fc90d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:36.348130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f85bd2-66fd-4033-96f9-9dc0053fc90d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:36.380502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618c617-8981-4dd0-96a7-841c1c4a6141 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:36.420151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2713a31-19d9-49c4-895a-7e88b7b3bbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:36.421823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713a31-19d9-49c4-895a-7e88b7b3bbf5 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:36.422143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713a31-19d9-49c4-895a-7e88b7b3bbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:36.422437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713a31-19d9-49c4-895a-7e88b7b3bbf5 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:36.423538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2713a31-19d9-49c4-895a-7e88b7b3bbf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:36.425010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2713a31-19d9-49c4-895a-7e88b7b3bbf5 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:36.449799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 44/88] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:36.458647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f85bd2-66fd-4033-96f9-9dc0053fc90d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.465931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:36.508783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 45/89] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:39:36 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:36.509561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2713a31-19d9-49c4-895a-7e88b7b3bbf5 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.559025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f85bd2-66fd-4033-96f9-9dc0053fc90d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:36.592000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:36.599831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc1b7f-bb6e-49cf-9a53-04b83100b3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:36.600909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc1b7f-bb6e-49cf-9a53-04b83100b3e2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:36.601293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc1b7f-bb6e-49cf-9a53-04b83100b3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:36.601573 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc1b7f-bb6e-49cf-9a53-04b83100b3e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:36.648766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2713a31-19d9-49c4-895a-7e88b7b3bbf5 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:36.681367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc1b7f-bb6e-49cf-9a53-04b83100b3e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:36.682922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc1b7f-bb6e-49cf-9a53-04b83100b3e2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:36.693267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8667d-931e-4934-bf03-c0b23e72a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:36.694379 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8667d-931e-4934-bf03-c0b23e72a3ad None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:36.694849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8667d-931e-4934-bf03-c0b23e72a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:36.695891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8667d-931e-4934-bf03-c0b23e72a3ad None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:36.696345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8667d-931e-4934-bf03-c0b23e72a3ad None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:36.697812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8667d-931e-4934-bf03-c0b23e72a3ad None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:36.700244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:36.701355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 46/90] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:36 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:36.702348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8667d-931e-4934-bf03-c0b23e72a3ad None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.825180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 45/91] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:36.826095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc1b7f-bb6e-49cf-9a53-04b83100b3e2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.853333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8667d-931e-4934-bf03-c0b23e72a3ad None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:36.890637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:36.911492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b5639-60ac-425e-801e-b7850a720c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:36.912449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b5639-60ac-425e-801e-b7850a720c6b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:36.912929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b5639-60ac-425e-801e-b7850a720c6b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:36.913616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b5639-60ac-425e-801e-b7850a720c6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:36.915384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b5639-60ac-425e-801e-b7850a720c6b None admin] RBAC: Adding query filter params (role.id=5dce8d92cffa40f79dfa1c03724c47e2, scope.project.id=56fabc309a30484ebda9a05789fa5271, user.id=73a2be1f4ead4bf0a051368674237133) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:36.916229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b5639-60ac-425e-801e-b7850a720c6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:36.919408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b5639-60ac-425e-801e-b7850a720c6b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:36.927210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 47/92] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:36 2022] GET /identity/v3/role_assignments?user.id=73a2be1f4ead4bf0a051368674237133&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:36.932459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b5639-60ac-425e-801e-b7850a720c6b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.997651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc1b7f-bb6e-49cf-9a53-04b83100b3e2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:37.021418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:37.036417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d58b1-f452-47ae-a969-38b746b376fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:37.037444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d58b1-f452-47ae-a969-38b746b376fb None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.037784 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d58b1-f452-47ae-a969-38b746b376fb None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.038121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d58b1-f452-47ae-a969-38b746b376fb None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.041141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d58b1-f452-47ae-a969-38b746b376fb None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:37.041259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d58b1-f452-47ae-a969-38b746b376fb None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:37.043154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 46/93] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:36 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:37.044799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d58b1-f452-47ae-a969-38b746b376fb None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:37.144426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b5639-60ac-425e-801e-b7850a720c6b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:37.188168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f3ab9-1676-4a40-b7ab-75ba8b54bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:37.189098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f3ab9-1676-4a40-b7ab-75ba8b54bbfa None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.189409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f3ab9-1676-4a40-b7ab-75ba8b54bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.190932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f3ab9-1676-4a40-b7ab-75ba8b54bbfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.191805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f3ab9-1676-4a40-b7ab-75ba8b54bbfa None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=56fabc309a30484ebda9a05789fa5271, user.id=16be6f6629504bf5b97a9530841ecf24) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:37.192124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f3ab9-1676-4a40-b7ab-75ba8b54bbfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:37.193604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477f3ab9-1676-4a40-b7ab-75ba8b54bbfa None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:37.199892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 48/94] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:36 2022] GET /identity/v3/role_assignments?user.id=16be6f6629504bf5b97a9530841ecf24&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=7a833c45603a4830b888e58e1232712a => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:37.201317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f3ab9-1676-4a40-b7ab-75ba8b54bbfa None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:37.222568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d58b1-f452-47ae-a969-38b746b376fb None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:37.261179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:37.315795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477f3ab9-1676-4a40-b7ab-75ba8b54bbfa None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:37.320292 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520902d-9191-4c70-9d4f-801cc3ea5ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:37.321208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520902d-9191-4c70-9d4f-801cc3ea5ea3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.321495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520902d-9191-4c70-9d4f-801cc3ea5ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.321776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520902d-9191-4c70-9d4f-801cc3ea5ea3 None admin] PATH_INFO: `/v3/projects/2dfec521565f46d1ac26beec7bbdd02b` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.326897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520902d-9191-4c70-9d4f-801cc3ea5ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=2dfec521565f46d1ac26beec7bbdd02b)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:37.328394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520902d-9191-4c70-9d4f-801cc3ea5ea3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:37.330579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 47/95] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:37 2022] GET /identity/v3/projects/2dfec521565f46d1ac26beec7bbdd02b => generated 312 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:37.336241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520902d-9191-4c70-9d4f-801cc3ea5ea3 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:37.354411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b13fb-34dc-4991-bf12-d315b5acef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:37.355254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b13fb-34dc-4991-bf12-d315b5acef1e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.355601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b13fb-34dc-4991-bf12-d315b5acef1e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.355887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b13fb-34dc-4991-bf12-d315b5acef1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.356835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b13fb-34dc-4991-bf12-d315b5acef1e None admin] RBAC: Adding query filter params (role.id=73a019c7fb8340a58c7dfe4550341949, scope.project.id=56fabc309a30484ebda9a05789fa5271, user.id=73a2be1f4ead4bf0a051368674237133) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:37.357159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b13fb-34dc-4991-bf12-d315b5acef1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:37.358833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b13fb-34dc-4991-bf12-d315b5acef1e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:37.365259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 49/96] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:37 2022] GET /identity/v3/role_assignments?user.id=73a2be1f4ead4bf0a051368674237133&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=73a019c7fb8340a58c7dfe4550341949 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:37.442653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:37.514288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520902d-9191-4c70-9d4f-801cc3ea5ea3 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:37.567246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e010b77-4e2c-4573-af0e-b4dec979b40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:37.568266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e010b77-4e2c-4573-af0e-b4dec979b40e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.568755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e010b77-4e2c-4573-af0e-b4dec979b40e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.569237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e010b77-4e2c-4573-af0e-b4dec979b40e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.570306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e010b77-4e2c-4573-af0e-b4dec979b40e None admin] RBAC: Adding query filter params (role.id=5dce8d92cffa40f79dfa1c03724c47e2, scope.project.id=2dfec521565f46d1ac26beec7bbdd02b, user.id=73a2be1f4ead4bf0a051368674237133) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:37.570845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e010b77-4e2c-4573-af0e-b4dec979b40e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:37.572501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e010b77-4e2c-4573-af0e-b4dec979b40e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:37.579265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 48/97] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:37 2022] GET /identity/v3/role_assignments?user.id=73a2be1f4ead4bf0a051368674237133&scope.project.id=2dfec521565f46d1ac26beec7bbdd02b&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:39.130317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040daf2-6446-4439-8c7c-8001bb066dc4 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.130848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040daf2-6446-4439-8c7c-8001bb066dc4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.130921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040daf2-6446-4439-8c7c-8001bb066dc4 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.132428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 50/98] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:39.138254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7379c-e375-4a54-bfc3-e57dab846d10 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.138927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7379c-e375-4a54-bfc3-e57dab846d10 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.139107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7379c-e375-4a54-bfc3-e57dab846d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.159310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-17f7379c-e375-4a54-bfc3-e57dab846d10 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:39.160438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-17f7379c-e375-4a54-bfc3-e57dab846d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:39.245414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 49/99] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:39.269099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040daf2-6446-4439-8c7c-8001bb066dc4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.271580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f35dab-b5e3-4d34-a37b-0bf515c948aa None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.271874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f35dab-b5e3-4d34-a37b-0bf515c948aa None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.272159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f35dab-b5e3-4d34-a37b-0bf515c948aa None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.275138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 50/100] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:39:39.280903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d71b-d178-48bc-9149-56bc8bfe77a1 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.281810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d71b-d178-48bc-9149-56bc8bfe77a1 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.282098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d71b-d178-48bc-9149-56bc8bfe77a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.301023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4d27d71b-d178-48bc-9149-56bc8bfe77a1 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:39.302212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4d27d71b-d178-48bc-9149-56bc8bfe77a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:39.350496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:39.373516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 51/101] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:39:39.402002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27d71b-d178-48bc-9149-56bc8bfe77a1 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.499258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040daf2-6446-4439-8c7c-8001bb066dc4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.540388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:39.587726 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fac3f3-03b8-4ba6-ab4e-dc3e674148b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.588919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fac3f3-03b8-4ba6-ab4e-dc3e674148b9 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.589427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fac3f3-03b8-4ba6-ab4e-dc3e674148b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.589898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fac3f3-03b8-4ba6-ab4e-dc3e674148b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.595176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fac3f3-03b8-4ba6-ab4e-dc3e674148b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:39.596839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fac3f3-03b8-4ba6-ab4e-dc3e674148b9 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:39.599034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 51/102] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:39.600731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27d71b-d178-48bc-9149-56bc8bfe77a1 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.604129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fac3f3-03b8-4ba6-ab4e-dc3e674148b9 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.679185 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba8b9-c284-415b-8b5c-d1e2da869949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.680008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba8b9-c284-415b-8b5c-d1e2da869949 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.680297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba8b9-c284-415b-8b5c-d1e2da869949 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.680580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba8b9-c284-415b-8b5c-d1e2da869949 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.711312 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ba8b9-c284-415b-8b5c-d1e2da869949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-f46ba8b9-c284-415b-8b5c-d1e2da869949 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:39.719192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:39.720051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:39.720830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 52/103] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:39:39.728364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba8b9-c284-415b-8b5c-d1e2da869949 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.784322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:39.844111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fac3f3-03b8-4ba6-ab4e-dc3e674148b9 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.913672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ba8b9-c284-415b-8b5c-d1e2da869949 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.955761 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49898aa7-74fb-457f-9cbb-0df87c514af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.956449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49898aa7-74fb-457f-9cbb-0df87c514af7 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.956739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49898aa7-74fb-457f-9cbb-0df87c514af7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.957015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49898aa7-74fb-457f-9cbb-0df87c514af7 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.957742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49898aa7-74fb-457f-9cbb-0df87c514af7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:39.959212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49898aa7-74fb-457f-9cbb-0df87c514af7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:39.975197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8dc53-997c-4ab5-aa29-0f57b27f28e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.975902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8dc53-997c-4ab5-aa29-0f57b27f28e1 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.976195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8dc53-997c-4ab5-aa29-0f57b27f28e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.976482 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8dc53-997c-4ab5-aa29-0f57b27f28e1 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.977259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8dc53-997c-4ab5-aa29-0f57b27f28e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:39.977574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8dc53-997c-4ab5-aa29-0f57b27f28e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:39.979025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8dc53-997c-4ab5-aa29-0f57b27f28e1 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:39.992549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 53/104] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:39.999010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8dc53-997c-4ab5-aa29-0f57b27f28e1 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:40.103316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 52/105] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:39:39 2022] POST /identity/v3/projects => generated 302 bytes in 499 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:39:40.105046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad5485-b192-4708-8d66-2700c9500f6b None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.105531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad5485-b192-4708-8d66-2700c9500f6b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.106050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad5485-b192-4708-8d66-2700c9500f6b None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.108425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 53/106] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:40.114601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32e5b-e2d7-448a-adc8-72570e526914 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.115220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32e5b-e2d7-448a-adc8-72570e526914 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.115698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32e5b-e2d7-448a-adc8-72570e526914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.118508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:40.129744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8dc53-997c-4ab5-aa29-0f57b27f28e1 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:40.155671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-d4f32e5b-e2d7-448a-adc8-72570e526914 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:40.157609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d4f32e5b-e2d7-448a-adc8-72570e526914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:40.179346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443e82-7a01-452f-ab85-7ed6b170b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:40.180258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443e82-7a01-452f-ab85-7ed6b170b16f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.180544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443e82-7a01-452f-ab85-7ed6b170b16f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.180824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443e82-7a01-452f-ab85-7ed6b170b16f None admin] PATH_INFO: `/v3/users/9fa09b4ccdb44d84b07ad48988be75aa` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.203349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443e82-7a01-452f-ab85-7ed6b170b16f None admin] RBAC: Authorizing `identity:get_user(user_id=9fa09b4ccdb44d84b07ad48988be75aa)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:40.204904 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443e82-7a01-452f-ab85-7ed6b170b16f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:40.207085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 54/107] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:39 2022] GET /identity/v3/users/9fa09b4ccdb44d84b07ad48988be75aa => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:40.213048 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443e82-7a01-452f-ab85-7ed6b170b16f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:40.244602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:40.353015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443e82-7a01-452f-ab85-7ed6b170b16f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:40.361179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 54/108] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:40.399336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcd261-3189-4c9c-9062-ff2d46905aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:40.399995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcd261-3189-4c9c-9062-ff2d46905aec None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.400279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcd261-3189-4c9c-9062-ff2d46905aec None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.400557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcd261-3189-4c9c-9062-ff2d46905aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.421432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f32e5b-e2d7-448a-adc8-72570e526914 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:40.426804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:40.478765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcd261-3189-4c9c-9062-ff2d46905aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:40.480643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcd261-3189-4c9c-9062-ff2d46905aec None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:40.652974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 55/109] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:40.655212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eac9f8-f740-48b6-a861-c8d260633036 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.655531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eac9f8-f740-48b6-a861-c8d260633036 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.655810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eac9f8-f740-48b6-a861-c8d260633036 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.657274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 56/110] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:40.662575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa787c3-2c2b-4b36-b54f-858eba3ddffe None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.662889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa787c3-2c2b-4b36-b54f-858eba3ddffe None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.663360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa787c3-2c2b-4b36-b54f-858eba3ddffe None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.664543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 57/111] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:40.666411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f32e5b-e2d7-448a-adc8-72570e526914 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:40.669605 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa787c3-2c2b-4b36-b54f-858eba3ddffe None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:40.702001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadb708-363a-40d8-901c-afdf3bccf48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:40.703180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadb708-363a-40d8-901c-afdf3bccf48f None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.703663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadb708-363a-40d8-901c-afdf3bccf48f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.704129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadb708-363a-40d8-901c-afdf3bccf48f None admin] PATH_INFO: `/v3/roles/member` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.714892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dadb708-363a-40d8-901c-afdf3bccf48f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-9dadb708-363a-40d8-901c-afdf3bccf48f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:40.723440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:40.724325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:40.725112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:40.727532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 55/112] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:39:40.729011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf63d18-6d82-482a-ae1a-ea635e1912f8 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.729526 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf63d18-6d82-482a-ae1a-ea635e1912f8 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.729661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:40.729806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf63d18-6d82-482a-ae1a-ea635e1912f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.747268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ecf63d18-6d82-482a-ae1a-ea635e1912f8 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:40.748555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ecf63d18-6d82-482a-ae1a-ea635e1912f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:40.770306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:40.779658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa787c3-2c2b-4b36-b54f-858eba3ddffe None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:40.814705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f826cf-a1a4-419a-a5bf-7cc3d39aaf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:40.815606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f826cf-a1a4-419a-a5bf-7cc3d39aaf77 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.815888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f826cf-a1a4-419a-a5bf-7cc3d39aaf77 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.816162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f826cf-a1a4-419a-a5bf-7cc3d39aaf77 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.817346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f826cf-a1a4-419a-a5bf-7cc3d39aaf77 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:40.818840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f826cf-a1a4-419a-a5bf-7cc3d39aaf77 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:40.820556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 58/113] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:40 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:40.829888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 56/114] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:40.829888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf63d18-6d82-482a-ae1a-ea635e1912f8 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:40.830063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b748b-8adb-40f0-bba9-2c4d0ef4b540 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.830063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b748b-8adb-40f0-bba9-2c4d0ef4b540 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.830063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b748b-8adb-40f0-bba9-2c4d0ef4b540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.853366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-ee4b748b-8adb-40f0-bba9-2c4d0ef4b540 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:40.854880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ee4b748b-8adb-40f0-bba9-2c4d0ef4b540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:40.926079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 59/115] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:40.927628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b1882-3c14-47c7-8b43-58ee043af2b2 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.928216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b1882-3c14-47c7-8b43-58ee043af2b2 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.928636 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b1882-3c14-47c7-8b43-58ee043af2b2 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.930143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 60/116] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:40.935368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b1882-3c14-47c7-8b43-58ee043af2b2 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:40.946181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf63d18-6d82-482a-ae1a-ea635e1912f8 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:40.980831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7ab33-5ecf-44b2-8e5f-097f74d4f504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:40.981607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7ab33-5ecf-44b2-8e5f-097f74d4f504 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.981895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7ab33-5ecf-44b2-8e5f-097f74d4f504 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.982174 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7ab33-5ecf-44b2-8e5f-097f74d4f504 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.982975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7ab33-5ecf-44b2-8e5f-097f74d4f504 None admin] RBAC: Adding query filter params (name=member) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:40.983286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7ab33-5ecf-44b2-8e5f-097f74d4f504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:40.984685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7ab33-5ecf-44b2-8e5f-097f74d4f504 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:40.992853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 57/117] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:40.993663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7ab33-5ecf-44b2-8e5f-097f74d4f504 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:40.998892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:41.015599 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:41.050438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b1882-3c14-47c7-8b43-58ee043af2b2 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:41.089506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57726d7-ce68-4ab8-8612-5ebe421db416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:41.090414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57726d7-ce68-4ab8-8612-5ebe421db416 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.090414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57726d7-ce68-4ab8-8612-5ebe421db416 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.090777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57726d7-ce68-4ab8-8612-5ebe421db416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.091612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57726d7-ce68-4ab8-8612-5ebe421db416 None admin] RBAC: Adding query filter params (role.id=abb740756b544900822d2326908df69e, scope.project.id=56fabc309a30484ebda9a05789fa5271, user.id=9fa09b4ccdb44d84b07ad48988be75aa) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:41.091923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57726d7-ce68-4ab8-8612-5ebe421db416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:41.093370 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57726d7-ce68-4ab8-8612-5ebe421db416 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:41.098966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 61/118] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:40 2022] GET /identity/v3/role_assignments?user.id=9fa09b4ccdb44d84b07ad48988be75aa&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=abb740756b544900822d2326908df69e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:41.099700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7ab33-5ecf-44b2-8e5f-097f74d4f504 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:41.108741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0d4c1-5dab-47fc-939a-d06ee3312b12 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.109377 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0d4c1-5dab-47fc-939a-d06ee3312b12 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.109884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0d4c1-5dab-47fc-939a-d06ee3312b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.134563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4ae0d4c1-5dab-47fc-939a-d06ee3312b12 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:41.136785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4ae0d4c1-5dab-47fc-939a-d06ee3312b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:41.143031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c6ffa-5c8e-4402-aa0d-28cc264a1af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:41.143849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c6ffa-5c8e-4402-aa0d-28cc264a1af9 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.144134 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c6ffa-5c8e-4402-aa0d-28cc264a1af9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.146877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c6ffa-5c8e-4402-aa0d-28cc264a1af9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.162385 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c6ffa-5c8e-4402-aa0d-28cc264a1af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-509c6ffa-5c8e-4402-aa0d-28cc264a1af9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:41.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:41.172798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:41.173564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:41.176070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 58/119] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:39:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:41.177161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c6ffa-5c8e-4402-aa0d-28cc264a1af9 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:41.230324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 62/120] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:41.232687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0d4c1-5dab-47fc-939a-d06ee3312b12 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:41.298951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:41.305477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:41.338791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c6ffa-5c8e-4402-aa0d-28cc264a1af9 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:41.367070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0d4c1-5dab-47fc-939a-d06ee3312b12 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:41.385107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cff410-ef64-4209-9c05-376f687dd7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:41.386391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cff410-ef64-4209-9c05-376f687dd7d8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.387042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cff410-ef64-4209-9c05-376f687dd7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.388176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cff410-ef64-4209-9c05-376f687dd7d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.405025 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cff410-ef64-4209-9c05-376f687dd7d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:41.411515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01532f53-5a49-41a1-8347-780555a68508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:41.412629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532f53-5a49-41a1-8347-780555a68508 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.412927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532f53-5a49-41a1-8347-780555a68508 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.413274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532f53-5a49-41a1-8347-780555a68508 None admin] PATH_INFO: `/v3/users/73a2be1f4ead4bf0a051368674237133` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-60cff410-ef64-4209-9c05-376f687dd7d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:41.414388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:41.415430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:41.416199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:41.416783 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01532f53-5a49-41a1-8347-780555a68508 None admin] RBAC: Authorizing `identity:get_user(user_id=73a2be1f4ead4bf0a051368674237133)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:41.416783 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01532f53-5a49-41a1-8347-780555a68508 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:41.418617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 59/121] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:39:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:41.419780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cff410-ef64-4209-9c05-376f687dd7d8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:41.422616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 63/122] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:41 2022] GET /identity/v3/users/73a2be1f4ead4bf0a051368674237133 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:41.424209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01532f53-5a49-41a1-8347-780555a68508 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:41.545929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01532f53-5a49-41a1-8347-780555a68508 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:41.558741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cff410-ef64-4209-9c05-376f687dd7d8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:41.564403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:41.590236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5383d-921d-4c58-aaed-0c1537cd5765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:41.591962 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5383d-921d-4c58-aaed-0c1537cd5765 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.592401 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5383d-921d-4c58-aaed-0c1537cd5765 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.593362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5383d-921d-4c58-aaed-0c1537cd5765 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.602809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60319fb-250f-422a-b1de-21109c244a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:41.603792 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60319fb-250f-422a-b1de-21109c244a78 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.604180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60319fb-250f-422a-b1de-21109c244a78 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.604565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60319fb-250f-422a-b1de-21109c244a78 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.605703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60319fb-250f-422a-b1de-21109c244a78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:41.606142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60319fb-250f-422a-b1de-21109c244a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:41.607945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60319fb-250f-422a-b1de-21109c244a78 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:41.608245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5383d-921d-4c58-aaed-0c1537cd5765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:41.613940 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:41.617991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 60/123] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-f7c5383d-921d-4c58-aaed-0c1537cd5765 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:41.619707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:41.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:41.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:41.623813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 64/124] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:41.623921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60319fb-250f-422a-b1de-21109c244a78 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:41.624916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5383d-921d-4c58-aaed-0c1537cd5765 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:41.752974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5383d-921d-4c58-aaed-0c1537cd5765 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:41.755374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60319fb-250f-422a-b1de-21109c244a78 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:41.787896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619d515-5309-46c6-8b26-3c4d0466e59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:41.788582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619d515-5309-46c6-8b26-3c4d0466e59f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.788871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619d515-5309-46c6-8b26-3c4d0466e59f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.789155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619d515-5309-46c6-8b26-3c4d0466e59f None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.789922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619d515-5309-46c6-8b26-3c4d0466e59f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:41.790222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619d515-5309-46c6-8b26-3c4d0466e59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:41.791813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619d515-5309-46c6-8b26-3c4d0466e59f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:41.795485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c785bc-acbc-42ff-b9fe-61a4dbcabb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:41.796630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c785bc-acbc-42ff-b9fe-61a4dbcabb03 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.797213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c785bc-acbc-42ff-b9fe-61a4dbcabb03 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.797715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c785bc-acbc-42ff-b9fe-61a4dbcabb03 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.799126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c785bc-acbc-42ff-b9fe-61a4dbcabb03 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:41.800535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 65/125] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:39:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:41.801186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619d515-5309-46c6-8b26-3c4d0466e59f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:41.801545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c785bc-acbc-42ff-b9fe-61a4dbcabb03 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:41.808706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 61/126] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:41 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:41.810349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c785bc-acbc-42ff-b9fe-61a4dbcabb03 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:41.838268 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:41.883664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:41.921505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619d515-5309-46c6-8b26-3c4d0466e59f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:41.923314 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c785bc-acbc-42ff-b9fe-61a4dbcabb03 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:41.961318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5694635-d1c8-45fe-bd18-894757f91474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:41.961512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4f047-86df-4e2d-8247-7db03dca5253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:41.962173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4f047-86df-4e2d-8247-7db03dca5253 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.962462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4f047-86df-4e2d-8247-7db03dca5253 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.962767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4f047-86df-4e2d-8247-7db03dca5253 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.963229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694635-d1c8-45fe-bd18-894757f91474 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.963534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4f047-86df-4e2d-8247-7db03dca5253 None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:41.963845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4f047-86df-4e2d-8247-7db03dca5253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:41.964050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694635-d1c8-45fe-bd18-894757f91474 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.964492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694635-d1c8-45fe-bd18-894757f91474 None admin] PATH_INFO: `/v3/users/16be6f6629504bf5b97a9530841ecf24` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.965245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b4f047-86df-4e2d-8247-7db03dca5253 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:41.966082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5694635-d1c8-45fe-bd18-894757f91474 None admin] RBAC: Authorizing `identity:get_user(user_id=16be6f6629504bf5b97a9530841ecf24)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:41.967950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5694635-d1c8-45fe-bd18-894757f91474 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:41.970100 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 62/127] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:41 2022] GET /identity/v3/users/16be6f6629504bf5b97a9530841ecf24 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:41.971145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5694635-d1c8-45fe-bd18-894757f91474 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:41.978418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 66/128] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:41.983366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4f047-86df-4e2d-8247-7db03dca5253 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:42.081593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:42.085355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4f047-86df-4e2d-8247-7db03dca5253 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:42.086844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5694635-d1c8-45fe-bd18-894757f91474 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:42.120222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911a7ff-8cad-49dd-a142-ca09eff2b2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:42.121219 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911a7ff-8cad-49dd-a142-ca09eff2b2d5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:42.121505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911a7ff-8cad-49dd-a142-ca09eff2b2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:42.122600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911a7ff-8cad-49dd-a142-ca09eff2b2d5 None admin] PATH_INFO: `/v3/users/73a2be1f4ead4bf0a051368674237133` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:42.124717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911a7ff-8cad-49dd-a142-ca09eff2b2d5 None admin] RBAC: Authorizing `identity:get_user(user_id=73a2be1f4ead4bf0a051368674237133)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:42.126168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bc9db-dd16-4f9a-bc50-965e18f42dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:42.126794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911a7ff-8cad-49dd-a142-ca09eff2b2d5 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:42.127512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bc9db-dd16-4f9a-bc50-965e18f42dd5 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:42.128119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bc9db-dd16-4f9a-bc50-965e18f42dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:42.128837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 67/129] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:41 2022] GET /identity/v3/users/73a2be1f4ead4bf0a051368674237133 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:42.129037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bc9db-dd16-4f9a-bc50-965e18f42dd5 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271/users/73a2be1f4ead4bf0a051368674237133/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:42.129649 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911a7ff-8cad-49dd-a142-ca09eff2b2d5 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:42.131617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9bc9db-dd16-4f9a-bc50-965e18f42dd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=56fabc309a30484ebda9a05789fa5271, user_id=73a2be1f4ead4bf0a051368674237133, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:42.133734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9bc9db-dd16-4f9a-bc50-965e18f42dd5 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:42.138772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:42.228912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 63/130] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:39:41 2022] PUT /identity/v3/projects/56fabc309a30484ebda9a05789fa5271/users/73a2be1f4ead4bf0a051368674237133/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:42.234818 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911a7ff-8cad-49dd-a142-ca09eff2b2d5 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:42.236462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bc9db-dd16-4f9a-bc50-965e18f42dd5 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:42.276081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a77717-8ba3-4611-862a-9107e0a30378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:42.276963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a77717-8ba3-4611-862a-9107e0a30378 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:42.277252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a77717-8ba3-4611-862a-9107e0a30378 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:42.277616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a77717-8ba3-4611-862a-9107e0a30378 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:42.279469 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a77717-8ba3-4611-862a-9107e0a30378 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:42.280929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a77717-8ba3-4611-862a-9107e0a30378 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:42.286516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 68/131] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:42 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:42.286516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a77717-8ba3-4611-862a-9107e0a30378 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:42.368733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:42.447280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bc9db-dd16-4f9a-bc50-965e18f42dd5 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:42.468740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a77717-8ba3-4611-862a-9107e0a30378 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:42.490192 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1081f8-b6a8-4b6c-95bb-ef55260475c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:42.491111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1081f8-b6a8-4b6c-95bb-ef55260475c2 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:42.491398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1081f8-b6a8-4b6c-95bb-ef55260475c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:42.492114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1081f8-b6a8-4b6c-95bb-ef55260475c2 None admin] PATH_INFO: `/v3/users/73a2be1f4ead4bf0a051368674237133` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:42.493596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1081f8-b6a8-4b6c-95bb-ef55260475c2 None admin] RBAC: Authorizing `identity:get_user(user_id=73a2be1f4ead4bf0a051368674237133)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:42.497709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1081f8-b6a8-4b6c-95bb-ef55260475c2 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:42.498071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 64/132] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:42 2022] GET /identity/v3/users/73a2be1f4ead4bf0a051368674237133 => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:42.502951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1081f8-b6a8-4b6c-95bb-ef55260475c2 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:42.510494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce16e30-776f-4098-bc69-aecb6d83e8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:42.511583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce16e30-776f-4098-bc69-aecb6d83e8ab None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:42.512135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce16e30-776f-4098-bc69-aecb6d83e8ab None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:42.512612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce16e30-776f-4098-bc69-aecb6d83e8ab None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:42.514238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce16e30-776f-4098-bc69-aecb6d83e8ab None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:42.515926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce16e30-776f-4098-bc69-aecb6d83e8ab None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:42.517804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 69/133] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:42 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:42.519176 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0889a8-a177-409d-a5b7-1b302cdbd43a None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:42.524024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0889a8-a177-409d-a5b7-1b302cdbd43a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:42.524587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0889a8-a177-409d-a5b7-1b302cdbd43a None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:42.526296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 70/134] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:42.527206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0889a8-a177-409d-a5b7-1b302cdbd43a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:42.576702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:42.620553 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1081f8-b6a8-4b6c-95bb-ef55260475c2 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:42.656469 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b974507-660a-4c40-9268-805c187b2fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:42.657377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b974507-660a-4c40-9268-805c187b2fa1 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:42.657671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b974507-660a-4c40-9268-805c187b2fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:42.657950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b974507-660a-4c40-9268-805c187b2fa1 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271/users/16be6f6629504bf5b97a9530841ecf24/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:42.659054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0889a8-a177-409d-a5b7-1b302cdbd43a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:42.660200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b974507-660a-4c40-9268-805c187b2fa1 None admin] RBAC: Authorizing `identity:create_grant(project_id=56fabc309a30484ebda9a05789fa5271, user_id=16be6f6629504bf5b97a9530841ecf24, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:42.661159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:42.661920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b974507-660a-4c40-9268-805c187b2fa1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:42.696856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b2f5c-3389-480e-84ad-1b3e3fdea865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:42.698099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b2f5c-3389-480e-84ad-1b3e3fdea865 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:42.698709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b2f5c-3389-480e-84ad-1b3e3fdea865 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:42.699227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b2f5c-3389-480e-84ad-1b3e3fdea865 None admin] PATH_INFO: `/v3/projects/2dfec521565f46d1ac26beec7bbdd02b` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:42.700809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b2f5c-3389-480e-84ad-1b3e3fdea865 None admin] RBAC: Authorizing `identity:get_project(project_id=2dfec521565f46d1ac26beec7bbdd02b)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:42.702720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b2f5c-3389-480e-84ad-1b3e3fdea865 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:42.704723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 71/135] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:42 2022] GET /identity/v3/projects/2dfec521565f46d1ac26beec7bbdd02b => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:42.705644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b2f5c-3389-480e-84ad-1b3e3fdea865 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:42.753320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 65/136] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:39:42 2022] PUT /identity/v3/projects/56fabc309a30484ebda9a05789fa5271/users/16be6f6629504bf5b97a9530841ecf24/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:42.762235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891cc94-7b6f-4c6e-8fe4-846a6cc281ba None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:42.762235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891cc94-7b6f-4c6e-8fe4-846a6cc281ba None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:42.762235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891cc94-7b6f-4c6e-8fe4-846a6cc281ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:42.794749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2891cc94-7b6f-4c6e-8fe4-846a6cc281ba None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:42.797755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2891cc94-7b6f-4c6e-8fe4-846a6cc281ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:42.904756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b2f5c-3389-480e-84ad-1b3e3fdea865 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:42.943922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c318c69-34f1-4bf6-80fa-b62f3e69e74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:42.944879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c318c69-34f1-4bf6-80fa-b62f3e69e74c None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:42.945172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c318c69-34f1-4bf6-80fa-b62f3e69e74c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:42.945453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c318c69-34f1-4bf6-80fa-b62f3e69e74c None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271/users/73a2be1f4ead4bf0a051368674237133/roles/73a019c7fb8340a58c7dfe4550341949` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:42.947453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 66/137] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:42.947871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c318c69-34f1-4bf6-80fa-b62f3e69e74c None admin] RBAC: Authorizing `identity:create_grant(project_id=56fabc309a30484ebda9a05789fa5271, user_id=73a2be1f4ead4bf0a051368674237133, role_id=73a019c7fb8340a58c7dfe4550341949)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:42.948286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891cc94-7b6f-4c6e-8fe4-846a6cc281ba None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:42.949485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c318c69-34f1-4bf6-80fa-b62f3e69e74c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:43.017801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:43.048730 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 72/138] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:39:42 2022] PUT /identity/v3/projects/56fabc309a30484ebda9a05789fa5271/users/73a2be1f4ead4bf0a051368674237133/roles/73a019c7fb8340a58c7dfe4550341949 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:43.049667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c318c69-34f1-4bf6-80fa-b62f3e69e74c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:43.075381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891cc94-7b6f-4c6e-8fe4-846a6cc281ba None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:43.120457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:43.137165 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a5ef6-3f55-4a22-aaaa-b4a7dd7c94af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:43.138124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a5ef6-3f55-4a22-aaaa-b4a7dd7c94af None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:43.138408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a5ef6-3f55-4a22-aaaa-b4a7dd7c94af None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:43.138975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a5ef6-3f55-4a22-aaaa-b4a7dd7c94af None admin] PATH_INFO: `/v3/projects/2dfec521565f46d1ac26beec7bbdd02b/users/73a2be1f4ead4bf0a051368674237133/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:43.143911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a5ef6-3f55-4a22-aaaa-b4a7dd7c94af None admin] RBAC: Authorizing `identity:create_grant(project_id=2dfec521565f46d1ac26beec7bbdd02b, user_id=73a2be1f4ead4bf0a051368674237133, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:43.145404 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a5ef6-3f55-4a22-aaaa-b4a7dd7c94af None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:43.225142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 67/139] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:39:42 2022] PUT /identity/v3/projects/2dfec521565f46d1ac26beec7bbdd02b/users/73a2be1f4ead4bf0a051368674237133/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:43.237934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c318c69-34f1-4bf6-80fa-b62f3e69e74c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:43.293612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3251c86-0a50-40bc-944c-35b6fdf701f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:43.295078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3251c86-0a50-40bc-944c-35b6fdf701f3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:43.295367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3251c86-0a50-40bc-944c-35b6fdf701f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:43.295652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3251c86-0a50-40bc-944c-35b6fdf701f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:43.297033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3251c86-0a50-40bc-944c-35b6fdf701f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:43.298568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3251c86-0a50-40bc-944c-35b6fdf701f3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:43.300391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 73/140] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:43.306002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a5ef6-3f55-4a22-aaaa-b4a7dd7c94af None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:43.439313 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a5ef6-3f55-4a22-aaaa-b4a7dd7c94af None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:43.454921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:43.460371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fa659-6ed7-421f-aaeb-958b61df735c None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:43.471155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fa659-6ed7-421f-aaeb-958b61df735c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:43.471155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fa659-6ed7-421f-aaeb-958b61df735c None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:43.471155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 74/141] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:43.471155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b5d6-a014-4c50-a7a0-6235ab01fe16 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:43.471155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b5d6-a014-4c50-a7a0-6235ab01fe16 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:43.471155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b5d6-a014-4c50-a7a0-6235ab01fe16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:43.500235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9223b5d6-a014-4c50-a7a0-6235ab01fe16 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:43.500235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9223b5d6-a014-4c50-a7a0-6235ab01fe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:43.504718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393d33b-db53-4d86-b97c-9cbcf18102fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:43.504718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393d33b-db53-4d86-b97c-9cbcf18102fe None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:43.504967 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393d33b-db53-4d86-b97c-9cbcf18102fe None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:43.505238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393d33b-db53-4d86-b97c-9cbcf18102fe None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:43.505987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393d33b-db53-4d86-b97c-9cbcf18102fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:43.507533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393d33b-db53-4d86-b97c-9cbcf18102fe None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:43.595989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 75/142] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:39:43.619698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b5d6-a014-4c50-a7a0-6235ab01fe16 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:43.640798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 68/143] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:39:43 2022] POST /identity/v3/users => generated 283 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:43.657385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:43.745455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b5d6-a014-4c50-a7a0-6235ab01fe16 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:43.787385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db64c3-4987-41fe-b0e1-9631c682e319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:43.788126 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db64c3-4987-41fe-b0e1-9631c682e319 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:43.788407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db64c3-4987-41fe-b0e1-9631c682e319 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:43.788680 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db64c3-4987-41fe-b0e1-9631c682e319 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:43.803770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db64c3-4987-41fe-b0e1-9631c682e319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-b4db64c3-4987-41fe-b0e1-9631c682e319 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:43.812207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:43.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:43.813959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 76/144] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:43.821033 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393d33b-db53-4d86-b97c-9cbcf18102fe None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:43.999144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393d33b-db53-4d86-b97c-9cbcf18102fe None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:44.075320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dabaa-e341-4710-ae2a-82b434ec3b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:44.075949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dabaa-e341-4710-ae2a-82b434ec3b42 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:44.076240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dabaa-e341-4710-ae2a-82b434ec3b42 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:44.076525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dabaa-e341-4710-ae2a-82b434ec3b42 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:44.077297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dabaa-e341-4710-ae2a-82b434ec3b42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:44.077612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dabaa-e341-4710-ae2a-82b434ec3b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:44.079051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dabaa-e341-4710-ae2a-82b434ec3b42 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:44.088138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 69/145] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:44.094246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db64c3-4987-41fe-b0e1-9631c682e319 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:44.224450 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:44.237054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db64c3-4987-41fe-b0e1-9631c682e319 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:44.284802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88a0b3-831b-4451-be85-ff84348b66b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:44.285695 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88a0b3-831b-4451-be85-ff84348b66b1 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:44.286021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88a0b3-831b-4451-be85-ff84348b66b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:44.286259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88a0b3-831b-4451-be85-ff84348b66b1 None admin] PATH_INFO: `/v3/users/9fa09b4ccdb44d84b07ad48988be75aa` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:44.287520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88a0b3-831b-4451-be85-ff84348b66b1 None admin] RBAC: Authorizing `identity:get_user(user_id=9fa09b4ccdb44d84b07ad48988be75aa)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:44.289056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88a0b3-831b-4451-be85-ff84348b66b1 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:44.291238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 77/146] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:44 2022] GET /identity/v3/users/9fa09b4ccdb44d84b07ad48988be75aa => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:44.296772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dabaa-e341-4710-ae2a-82b434ec3b42 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:44.336019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:44.425593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dabaa-e341-4710-ae2a-82b434ec3b42 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:44.472526 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f5221-fe2b-4a36-816a-6736800504c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:44.473409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5221-fe2b-4a36-816a-6736800504c7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:44.473693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5221-fe2b-4a36-816a-6736800504c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:44.473982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5221-fe2b-4a36-816a-6736800504c7 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:44.475230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f5221-fe2b-4a36-816a-6736800504c7 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:44.476693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f5221-fe2b-4a36-816a-6736800504c7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:44.478409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 70/147] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:44 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:44.484561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88a0b3-831b-4451-be85-ff84348b66b1 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:44.616632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88a0b3-831b-4451-be85-ff84348b66b1 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:44.661108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200459dd-6aa4-41d1-95c8-a7f2c923796e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:44.662017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200459dd-6aa4-41d1-95c8-a7f2c923796e None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:44.662301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200459dd-6aa4-41d1-95c8-a7f2c923796e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:44.662605 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200459dd-6aa4-41d1-95c8-a7f2c923796e None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271/users/9fa09b4ccdb44d84b07ad48988be75aa/roles/abb740756b544900822d2326908df69e` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:44.664602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200459dd-6aa4-41d1-95c8-a7f2c923796e None admin] RBAC: Authorizing `identity:create_grant(project_id=56fabc309a30484ebda9a05789fa5271, user_id=9fa09b4ccdb44d84b07ad48988be75aa, role_id=abb740756b544900822d2326908df69e)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:44.666169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200459dd-6aa4-41d1-95c8-a7f2c923796e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:44.752082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 78/148] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:39:44 2022] PUT /identity/v3/projects/56fabc309a30484ebda9a05789fa5271/users/9fa09b4ccdb44d84b07ad48988be75aa/roles/abb740756b544900822d2326908df69e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:45.043847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da1625-f87a-4715-97d1-29621ce94466 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:45.044238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da1625-f87a-4715-97d1-29621ce94466 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:45.044415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da1625-f87a-4715-97d1-29621ce94466 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:45.045962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 71/149] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:45.060388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1f00b-7161-4c80-8409-0b76a50a0b62 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:45.060712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1f00b-7161-4c80-8409-0b76a50a0b62 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:45.060960 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1f00b-7161-4c80-8409-0b76a50a0b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:45.084152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-cdb1f00b-7161-4c80-8409-0b76a50a0b62 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:45.085375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cdb1f00b-7161-4c80-8409-0b76a50a0b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:45.100793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:45.160021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 79/150] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:45.204009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da1625-f87a-4715-97d1-29621ce94466 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:45.322135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:45.326639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da1625-f87a-4715-97d1-29621ce94466 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:45.361499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7cf94-4969-4ccd-829e-d76f25a1cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:45.362306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7cf94-4969-4ccd-829e-d76f25a1cb67 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:45.362621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7cf94-4969-4ccd-829e-d76f25a1cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:45.362900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7cf94-4969-4ccd-829e-d76f25a1cb67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:45.375285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d7cf94-4969-4ccd-829e-d76f25a1cb67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-32d7cf94-4969-4ccd-829e-d76f25a1cb67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:45.383105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:45.383989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:45.384788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 72/151] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:45.392181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1f00b-7161-4c80-8409-0b76a50a0b62 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:45.550410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1f00b-7161-4c80-8409-0b76a50a0b62 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:45.592591 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b925b3-0682-416f-b6a1-4fa35a215cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:45.593349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b925b3-0682-416f-b6a1-4fa35a215cb2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:45.593640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b925b3-0682-416f-b6a1-4fa35a215cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:45.593915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b925b3-0682-416f-b6a1-4fa35a215cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:45.594697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b925b3-0682-416f-b6a1-4fa35a215cb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:45.595007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b925b3-0682-416f-b6a1-4fa35a215cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:45.596395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b925b3-0682-416f-b6a1-4fa35a215cb2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:45.610131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 80/152] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:45.615740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7cf94-4969-4ccd-829e-d76f25a1cb67 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:45.681939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bad052-070d-4728-8351-14f1c2193e25 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:45.682497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bad052-070d-4728-8351-14f1c2193e25 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:45.682936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bad052-070d-4728-8351-14f1c2193e25 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:45.685372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 81/153] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:39:45.690394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326d58-0164-4c76-bfd1-29c0aa10862a None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:45.690873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326d58-0164-4c76-bfd1-29c0aa10862a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:45.691315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326d58-0164-4c76-bfd1-29c0aa10862a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:45.719670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-49326d58-0164-4c76-bfd1-29c0aa10862a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:45.721088 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-49326d58-0164-4c76-bfd1-29c0aa10862a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:45.723382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:45.807654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7cf94-4969-4ccd-829e-d76f25a1cb67 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:45.889584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e2304-adea-452f-bb13-70cdf4460495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:45.890495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e2304-adea-452f-bb13-70cdf4460495 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:45.890778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e2304-adea-452f-bb13-70cdf4460495 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:45.891098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e2304-adea-452f-bb13-70cdf4460495 None admin] PATH_INFO: `/v3/users/73a2be1f4ead4bf0a051368674237133` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:45.895229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e2304-adea-452f-bb13-70cdf4460495 None admin] RBAC: Authorizing `identity:get_user(user_id=73a2be1f4ead4bf0a051368674237133)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:45.896434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 82/154] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:39:45.897228 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e2304-adea-452f-bb13-70cdf4460495 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:45.898966 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 73/155] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:45 2022] GET /identity/v3/users/73a2be1f4ead4bf0a051368674237133 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:45.907485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49326d58-0164-4c76-bfd1-29c0aa10862a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:45.962203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e2304-adea-452f-bb13-70cdf4460495 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:46.054896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:46.122211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e2304-adea-452f-bb13-70cdf4460495 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:46.152687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49326d58-0164-4c76-bfd1-29c0aa10862a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:46.203684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d244f1-88f0-4561-b6d2-9b01820bf364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:46.204913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d244f1-88f0-4561-b6d2-9b01820bf364 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:46.205382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d244f1-88f0-4561-b6d2-9b01820bf364 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:46.206025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d244f1-88f0-4561-b6d2-9b01820bf364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:46.228682 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:46.243226 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427a352-b261-4c75-8ef3-a21603f8b465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:46.244213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427a352-b261-4c75-8ef3-a21603f8b465 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:46.244606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427a352-b261-4c75-8ef3-a21603f8b465 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:46.244985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427a352-b261-4c75-8ef3-a21603f8b465 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:46.271776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427a352-b261-4c75-8ef3-a21603f8b465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-0427a352-b261-4c75-8ef3-a21603f8b465 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:46.292737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:46.294273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:46.295150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 74/156] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:39:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:46.300686 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427a352-b261-4c75-8ef3-a21603f8b465 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:46.302325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d244f1-88f0-4561-b6d2-9b01820bf364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:46.304148 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d244f1-88f0-4561-b6d2-9b01820bf364 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:46.435521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:46.447704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427a352-b261-4c75-8ef3-a21603f8b465 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:46.488387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b775917-b201-495c-aa92-ba4905f4ab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:46.489369 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b775917-b201-495c-aa92-ba4905f4ab49 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:46.492582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b775917-b201-495c-aa92-ba4905f4ab49 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:46.493025 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b775917-b201-495c-aa92-ba4905f4ab49 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:46.494007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b775917-b201-495c-aa92-ba4905f4ab49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:46.494501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b775917-b201-495c-aa92-ba4905f4ab49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:46.496070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b775917-b201-495c-aa92-ba4905f4ab49 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:46.509062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 75/157] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:46.516079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b775917-b201-495c-aa92-ba4905f4ab49 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:46.526418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 83/158] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:46.533488 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d244f1-88f0-4561-b6d2-9b01820bf364 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:46.669933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b775917-b201-495c-aa92-ba4905f4ab49 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:46.678809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:46.765119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597288b-7344-4977-a015-0fef5e854660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:46.766376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597288b-7344-4977-a015-0fef5e854660 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:46.766906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597288b-7344-4977-a015-0fef5e854660 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:46.767386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597288b-7344-4977-a015-0fef5e854660 None admin] PATH_INFO: `/v3/users/16be6f6629504bf5b97a9530841ecf24` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:46.771181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597288b-7344-4977-a015-0fef5e854660 None admin] RBAC: Authorizing `identity:get_user(user_id=16be6f6629504bf5b97a9530841ecf24)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:46.772891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597288b-7344-4977-a015-0fef5e854660 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:46.775231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 76/159] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:46 2022] GET /identity/v3/users/16be6f6629504bf5b97a9530841ecf24 => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:46.782155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597288b-7344-4977-a015-0fef5e854660 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:46.827640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:46.850120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d244f1-88f0-4561-b6d2-9b01820bf364 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:46.955196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7483f53-2c1f-4a1b-bca1-4ca8ff82ad5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:46.958504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7483f53-2c1f-4a1b-bca1-4ca8ff82ad5f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:46.958504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7483f53-2c1f-4a1b-bca1-4ca8ff82ad5f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:46.958504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7483f53-2c1f-4a1b-bca1-4ca8ff82ad5f None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:46.963248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7483f53-2c1f-4a1b-bca1-4ca8ff82ad5f None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:46.964763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7483f53-2c1f-4a1b-bca1-4ca8ff82ad5f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:46.967101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 84/160] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:46 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:46.968324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4e652-3a61-4d2f-86a0-9b334a4e3188 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:46.968608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4e652-3a61-4d2f-86a0-9b334a4e3188 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:46.968881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4e652-3a61-4d2f-86a0-9b334a4e3188 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:46.970144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 85/161] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:46.980173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcd8d0-be7d-4ef7-9c2b-574eb2c5b1bd None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:46.980457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcd8d0-be7d-4ef7-9c2b-574eb2c5b1bd None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:46.980922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcd8d0-be7d-4ef7-9c2b-574eb2c5b1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:46.999656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597288b-7344-4977-a015-0fef5e854660 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:47.034431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-8abcd8d0-be7d-4ef7-9c2b-574eb2c5b1bd None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:47.035605 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8abcd8d0-be7d-4ef7-9c2b-574eb2c5b1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:47.076036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7679bfe-aa00-4cb2-b9c0-8f013b28fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:47.077092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679bfe-aa00-4cb2-b9c0-8f013b28fcda None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:47.077579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679bfe-aa00-4cb2-b9c0-8f013b28fcda None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:47.078065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679bfe-aa00-4cb2-b9c0-8f013b28fcda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:47.172679 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 86/162] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:47.176288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcd8d0-be7d-4ef7-9c2b-574eb2c5b1bd None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:47.201933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7679bfe-aa00-4cb2-b9c0-8f013b28fcda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:47.203822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7679bfe-aa00-4cb2-b9c0-8f013b28fcda None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:47.234405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:47.261473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:47.365099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcd8d0-be7d-4ef7-9c2b-574eb2c5b1bd None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:47.379755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 77/163] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:39:47.381474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3df4b-b78d-4ef6-b426-f3ff558e4eb8 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:47.383711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3df4b-b78d-4ef6-b426-f3ff558e4eb8 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:47.384216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3df4b-b78d-4ef6-b426-f3ff558e4eb8 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:47.387853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 78/164] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:47.389098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3df4b-b78d-4ef6-b426-f3ff558e4eb8 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:47.436529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37078a4-1894-4f44-8a9e-44a622e209a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:47.437546 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37078a4-1894-4f44-8a9e-44a622e209a2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:47.438033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37078a4-1894-4f44-8a9e-44a622e209a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:47.438637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37078a4-1894-4f44-8a9e-44a622e209a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:47.440817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37078a4-1894-4f44-8a9e-44a622e209a2 None admin] RBAC: Adding query filter params (role.id=5dce8d92cffa40f79dfa1c03724c47e2, scope.project.id=56fabc309a30484ebda9a05789fa5271, user.id=73a2be1f4ead4bf0a051368674237133) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:47.442060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37078a4-1894-4f44-8a9e-44a622e209a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:47.443766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37078a4-1894-4f44-8a9e-44a622e209a2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:47.453149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 87/165] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:47 2022] GET /identity/v3/role_assignments?user.id=73a2be1f4ead4bf0a051368674237133&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:47.455304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37078a4-1894-4f44-8a9e-44a622e209a2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:47.630561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37078a4-1894-4f44-8a9e-44a622e209a2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:47.650744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:47.659751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3df4b-b78d-4ef6-b426-f3ff558e4eb8 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:47.670135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a0815-1f30-4b55-b5b2-fd4588e12ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:47.670135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a0815-1f30-4b55-b5b2-fd4588e12ff2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:47.670870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a0815-1f30-4b55-b5b2-fd4588e12ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:47.671339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a0815-1f30-4b55-b5b2-fd4588e12ff2 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:47.672772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a0815-1f30-4b55-b5b2-fd4588e12ff2 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:47.674432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a0815-1f30-4b55-b5b2-fd4588e12ff2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:47.676529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 88/166] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:47 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:47.677890 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0f628-8417-4d1f-a71a-98f678e90218 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:47.678371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0f628-8417-4d1f-a71a-98f678e90218 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:47.678978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0f628-8417-4d1f-a71a-98f678e90218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:47.697074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:47.703050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-b1c0f628-8417-4d1f-a71a-98f678e90218 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:47.704549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b1c0f628-8417-4d1f-a71a-98f678e90218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:47.746024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89424163-8605-406b-9026-96dc918fab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:47.747432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89424163-8605-406b-9026-96dc918fab6c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:47.747718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89424163-8605-406b-9026-96dc918fab6c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:47.747993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89424163-8605-406b-9026-96dc918fab6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:47.779314 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89424163-8605-406b-9026-96dc918fab6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-89424163-8605-406b-9026-96dc918fab6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:47.787296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:47.788201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:47.789204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 79/167] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:47.789863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89424163-8605-406b-9026-96dc918fab6c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:47.836305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 89/168] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:47.837429 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0f628-8417-4d1f-a71a-98f678e90218 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:48.025786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:48.035454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0f628-8417-4d1f-a71a-98f678e90218 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:48.072646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89424163-8605-406b-9026-96dc918fab6c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:48.091291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa55a0-a8db-4897-b3ce-0514c00973d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:48.092021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa55a0-a8db-4897-b3ce-0514c00973d2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.092305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa55a0-a8db-4897-b3ce-0514c00973d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.092581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa55a0-a8db-4897-b3ce-0514c00973d2 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.093396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa55a0-a8db-4897-b3ce-0514c00973d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:48.093704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa55a0-a8db-4897-b3ce-0514c00973d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:48.095145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa55a0-a8db-4897-b3ce-0514c00973d2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:48.105809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 90/169] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:48.107531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa55a0-a8db-4897-b3ce-0514c00973d2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:48.169715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4d98b-c49a-4ea7-b9da-9572b1404bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:48.170295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4d98b-c49a-4ea7-b9da-9572b1404bf8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.170597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4d98b-c49a-4ea7-b9da-9572b1404bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.170873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4d98b-c49a-4ea7-b9da-9572b1404bf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.171708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4d98b-c49a-4ea7-b9da-9572b1404bf8 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=56fabc309a30484ebda9a05789fa5271, user.id=16be6f6629504bf5b97a9530841ecf24) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:48.172016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4d98b-c49a-4ea7-b9da-9572b1404bf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:48.173469 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4d98b-c49a-4ea7-b9da-9572b1404bf8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:48.192498 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 80/170] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:47 2022] GET /identity/v3/role_assignments?user.id=16be6f6629504bf5b97a9530841ecf24&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=7a833c45603a4830b888e58e1232712a => generated 608 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:48.193347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4d98b-c49a-4ea7-b9da-9572b1404bf8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:48.239534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa55a0-a8db-4897-b3ce-0514c00973d2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:48.310728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:48.314688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74732b6-3035-4d02-b554-a2a143e69827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:48.315703 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74732b6-3035-4d02-b554-a2a143e69827 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.316158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74732b6-3035-4d02-b554-a2a143e69827 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.316575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74732b6-3035-4d02-b554-a2a143e69827 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.335873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74732b6-3035-4d02-b554-a2a143e69827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-e74732b6-3035-4d02-b554-a2a143e69827 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:48.348647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:48.349709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:48.350720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:48.352679 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 91/171] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:39:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:48.373171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74732b6-3035-4d02-b554-a2a143e69827 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:48.390323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4d98b-c49a-4ea7-b9da-9572b1404bf8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:48.422809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:48.465036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b9b18-5f3f-4df5-86e1-f37d536b8027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:48.466813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b9b18-5f3f-4df5-86e1-f37d536b8027 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.467238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b9b18-5f3f-4df5-86e1-f37d536b8027 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.467652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b9b18-5f3f-4df5-86e1-f37d536b8027 None admin] PATH_INFO: `/v3/users/73a2be1f4ead4bf0a051368674237133` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.470528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b9b18-5f3f-4df5-86e1-f37d536b8027 None admin] RBAC: Authorizing `identity:get_user(user_id=73a2be1f4ead4bf0a051368674237133)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:48.472387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b9b18-5f3f-4df5-86e1-f37d536b8027 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:48.479170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 81/172] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:48 2022] GET /identity/v3/users/73a2be1f4ead4bf0a051368674237133 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:48.484275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b9b18-5f3f-4df5-86e1-f37d536b8027 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:48.592102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74732b6-3035-4d02-b554-a2a143e69827 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:48.635577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b9b18-5f3f-4df5-86e1-f37d536b8027 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:48.642368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29490858-43d3-4ad1-99de-f3046377a120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:48.643082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29490858-43d3-4ad1-99de-f3046377a120 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.643365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29490858-43d3-4ad1-99de-f3046377a120 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.643643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29490858-43d3-4ad1-99de-f3046377a120 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.644408 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29490858-43d3-4ad1-99de-f3046377a120 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:48.644716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29490858-43d3-4ad1-99de-f3046377a120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:48.646137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29490858-43d3-4ad1-99de-f3046377a120 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:48.649800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:48.659576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 92/173] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:39:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:48.660926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097616bc-8cd9-4e84-a508-5429591fc853 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.663955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097616bc-8cd9-4e84-a508-5429591fc853 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.663955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097616bc-8cd9-4e84-a508-5429591fc853 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.663955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 93/174] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:48.668423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e8c60-75a2-4684-9fb5-51b56b577881 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.668724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e8c60-75a2-4684-9fb5-51b56b577881 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.669005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e8c60-75a2-4684-9fb5-51b56b577881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.678374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39749f-e137-44b6-853e-e9ee178891fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:48.679718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39749f-e137-44b6-853e-e9ee178891fe None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.680197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39749f-e137-44b6-853e-e9ee178891fe None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.680611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39749f-e137-44b6-853e-e9ee178891fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.703540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-3b6e8c60-75a2-4684-9fb5-51b56b577881 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:48.704746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3b6e8c60-75a2-4684-9fb5-51b56b577881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:48.757229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39749f-e137-44b6-853e-e9ee178891fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:48.758979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39749f-e137-44b6-853e-e9ee178891fe None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:48.823609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 94/175] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:48.825801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e8c60-75a2-4684-9fb5-51b56b577881 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:48.843953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:48.893582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 82/176] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:48.895422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f4bb-831a-444a-adcd-f8f0e1d0142a None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.896020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f4bb-831a-444a-adcd-f8f0e1d0142a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.896481 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f4bb-831a-444a-adcd-f8f0e1d0142a None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.898243 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 83/177] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:48.899227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32f4bb-831a-444a-adcd-f8f0e1d0142a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:48.913923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:48.944644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e8c60-75a2-4684-9fb5-51b56b577881 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:48.981117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debc964-9529-4727-af65-161cb9970775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:48.982412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debc964-9529-4727-af65-161cb9970775 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.982736 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debc964-9529-4727-af65-161cb9970775 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.983012 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debc964-9529-4727-af65-161cb9970775 None admin] PATH_INFO: `/v3/users/73a2be1f4ead4bf0a051368674237133` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.984309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debc964-9529-4727-af65-161cb9970775 None admin] RBAC: Authorizing `identity:get_user(user_id=73a2be1f4ead4bf0a051368674237133)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:48.985763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debc964-9529-4727-af65-161cb9970775 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:48.992062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 95/178] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:48 2022] GET /identity/v3/users/73a2be1f4ead4bf0a051368674237133 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:48.992062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5b429-55d4-4423-a3cd-0d59238d5d9a None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:48.992062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5b429-55d4-4423-a3cd-0d59238d5d9a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:48.992062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5b429-55d4-4423-a3cd-0d59238d5d9a None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:48.992062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 96/179] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:48.992062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5b429-55d4-4423-a3cd-0d59238d5d9a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:49.014308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32f4bb-831a-444a-adcd-f8f0e1d0142a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:49.048906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd895b8-8c59-4a24-b341-aea424e86a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:49.049697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd895b8-8c59-4a24-b341-aea424e86a66 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.049987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd895b8-8c59-4a24-b341-aea424e86a66 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.050261 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd895b8-8c59-4a24-b341-aea424e86a66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.060902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd895b8-8c59-4a24-b341-aea424e86a66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-5fd895b8-8c59-4a24-b341-aea424e86a66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:49.068730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:49.069626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:49.070430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 84/180] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:39:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:49.071921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade9f08-e714-41cf-9260-daa956ad1af2 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.071921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade9f08-e714-41cf-9260-daa956ad1af2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.071921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade9f08-e714-41cf-9260-daa956ad1af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.076596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:49.088151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-6ade9f08-e714-41cf-9260-daa956ad1af2 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:49.089338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6ade9f08-e714-41cf-9260-daa956ad1af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:49.096502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5b429-55d4-4423-a3cd-0d59238d5d9a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:49.130001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf3ccc-4c7c-4a13-972b-b8a74e6b09df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:49.130899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf3ccc-4c7c-4a13-972b-b8a74e6b09df None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.131184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf3ccc-4c7c-4a13-972b-b8a74e6b09df None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.131461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf3ccc-4c7c-4a13-972b-b8a74e6b09df None admin] PATH_INFO: `/v3/projects/2dfec521565f46d1ac26beec7bbdd02b` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.132635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf3ccc-4c7c-4a13-972b-b8a74e6b09df None admin] RBAC: Authorizing `identity:get_project(project_id=2dfec521565f46d1ac26beec7bbdd02b)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:49.134119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf3ccc-4c7c-4a13-972b-b8a74e6b09df None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:49.135847 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 97/181] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:48 2022] GET /identity/v3/projects/2dfec521565f46d1ac26beec7bbdd02b => generated 312 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:49.136600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf3ccc-4c7c-4a13-972b-b8a74e6b09df None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:49.152366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:49.156927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 85/182] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:49.159779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148ce41-5b6d-4e4f-a319-c7128857d72d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.160062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148ce41-5b6d-4e4f-a319-c7128857d72d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.160335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148ce41-5b6d-4e4f-a319-c7128857d72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.177231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-6148ce41-5b6d-4e4f-a319-c7128857d72d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:49.178881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6148ce41-5b6d-4e4f-a319-c7128857d72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:49.235473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf3ccc-4c7c-4a13-972b-b8a74e6b09df None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:49.244000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 86/183] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:49.245308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef00c20-9e44-4e8b-b51f-701cf4329260 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.245645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef00c20-9e44-4e8b-b51f-701cf4329260 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.245877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef00c20-9e44-4e8b-b51f-701cf4329260 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.247721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 87/184] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:49.252425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef00c20-9e44-4e8b-b51f-701cf4329260 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:49.271881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39f94d-1d1f-490d-962b-0f5967e6af53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:49.272527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39f94d-1d1f-490d-962b-0f5967e6af53 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.272806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39f94d-1d1f-490d-962b-0f5967e6af53 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.273081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39f94d-1d1f-490d-962b-0f5967e6af53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.290685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:49.338813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39f94d-1d1f-490d-962b-0f5967e6af53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:49.340295 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39f94d-1d1f-490d-962b-0f5967e6af53 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:49.352319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef00c20-9e44-4e8b-b51f-701cf4329260 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:49.381223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:49.385831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c2ed0-7c48-485e-b19d-6331e9c2298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:49.386587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c2ed0-7c48-485e-b19d-6331e9c2298b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.386929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c2ed0-7c48-485e-b19d-6331e9c2298b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.387910 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c2ed0-7c48-485e-b19d-6331e9c2298b None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.388857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c2ed0-7c48-485e-b19d-6331e9c2298b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:49.389163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c2ed0-7c48-485e-b19d-6331e9c2298b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:49.390562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c2ed0-7c48-485e-b19d-6331e9c2298b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:49.398441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 88/185] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:49.399313 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c2ed0-7c48-485e-b19d-6331e9c2298b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:49.442592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 98/186] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:49.443429 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39f94d-1d1f-490d-962b-0f5967e6af53 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:49.497355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:49.506287 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c2ed0-7c48-485e-b19d-6331e9c2298b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:49.538918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c6c2c-c026-4cb7-a375-84af7fe2ebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:49.539571 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c6c2c-c026-4cb7-a375-84af7fe2ebe1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.539853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c6c2c-c026-4cb7-a375-84af7fe2ebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.540126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c6c2c-c026-4cb7-a375-84af7fe2ebe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.540952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c6c2c-c026-4cb7-a375-84af7fe2ebe1 None admin] RBAC: Adding query filter params (role.id=5dce8d92cffa40f79dfa1c03724c47e2, scope.project.id=2dfec521565f46d1ac26beec7bbdd02b, user.id=73a2be1f4ead4bf0a051368674237133) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:49.541259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c6c2c-c026-4cb7-a375-84af7fe2ebe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:49.542719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c6c2c-c026-4cb7-a375-84af7fe2ebe1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:49.546462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39f94d-1d1f-490d-962b-0f5967e6af53 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:49.549172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 89/187] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:49 2022] GET /identity/v3/role_assignments?user.id=73a2be1f4ead4bf0a051368674237133&scope.project.id=2dfec521565f46d1ac26beec7bbdd02b&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:49.550327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4dedc-99fa-46b8-b394-5cb1fab40a0c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.550645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4dedc-99fa-46b8-b394-5cb1fab40a0c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.550920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4dedc-99fa-46b8-b394-5cb1fab40a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.574057 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-5fb4dedc-99fa-46b8-b394-5cb1fab40a0c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:49.575616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5fb4dedc-99fa-46b8-b394-5cb1fab40a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:49.586535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b747ad-611a-4f74-affe-fa3ac6a2b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:49.587619 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b747ad-611a-4f74-affe-fa3ac6a2b699 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.587619 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b747ad-611a-4f74-affe-fa3ac6a2b699 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.587900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b747ad-611a-4f74-affe-fa3ac6a2b699 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.589144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b747ad-611a-4f74-affe-fa3ac6a2b699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:49.590697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b747ad-611a-4f74-affe-fa3ac6a2b699 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:49.592515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 99/188] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:49.593133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b747ad-611a-4f74-affe-fa3ac6a2b699 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:49.641552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:49.659385 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 90/189] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:49.671572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4dedc-99fa-46b8-b394-5cb1fab40a0c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:49.704290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b747ad-611a-4f74-affe-fa3ac6a2b699 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:49.721800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:49.740513 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb04a5b-8494-4643-b40e-83b1f8fab31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:49.741324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb04a5b-8494-4643-b40e-83b1f8fab31a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.741604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb04a5b-8494-4643-b40e-83b1f8fab31a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.741879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb04a5b-8494-4643-b40e-83b1f8fab31a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.756795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb04a5b-8494-4643-b40e-83b1f8fab31a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-6fb04a5b-8494-4643-b40e-83b1f8fab31a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:49.766384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:49.767446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:49.768251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 100/190] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:49.768919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb04a5b-8494-4643-b40e-83b1f8fab31a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:49.794378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4dedc-99fa-46b8-b394-5cb1fab40a0c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:49.828672 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189e96c-1543-438b-8948-9c529da93a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:49.829550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189e96c-1543-438b-8948-9c529da93a26 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.829850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189e96c-1543-438b-8948-9c529da93a26 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.830121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189e96c-1543-438b-8948-9c529da93a26 None admin] PATH_INFO: `/v3/users/00eb2269a55046bd9c0fa851322f177c` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.844102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189e96c-1543-438b-8948-9c529da93a26 None admin] RBAC: Authorizing `identity:get_user(user_id=00eb2269a55046bd9c0fa851322f177c)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:49.845574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189e96c-1543-438b-8948-9c529da93a26 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:49.847426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 91/191] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:49 2022] GET /identity/v3/users/00eb2269a55046bd9c0fa851322f177c => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:49.848378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189e96c-1543-438b-8948-9c529da93a26 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:49.873494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb04a5b-8494-4643-b40e-83b1f8fab31a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:49.893081 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:49.906548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53051e66-fabb-423e-b45d-253d08316ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:49.907450 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53051e66-fabb-423e-b45d-253d08316ba3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.907731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53051e66-fabb-423e-b45d-253d08316ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.907996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53051e66-fabb-423e-b45d-253d08316ba3 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.909185 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53051e66-fabb-423e-b45d-253d08316ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:49.910670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53051e66-fabb-423e-b45d-253d08316ba3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:49.912547 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 101/192] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:49 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:49.913132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53051e66-fabb-423e-b45d-253d08316ba3 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:49.952045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189e96c-1543-438b-8948-9c529da93a26 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:49.956737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:49.986070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c21ea-7f3a-44d4-97a7-7679705006da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:49.986777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c21ea-7f3a-44d4-97a7-7679705006da None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:49.987065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c21ea-7f3a-44d4-97a7-7679705006da None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:49.987344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c21ea-7f3a-44d4-97a7-7679705006da None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:49.988067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c21ea-7f3a-44d4-97a7-7679705006da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:49.989514 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c21ea-7f3a-44d4-97a7-7679705006da None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:50.017109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53051e66-fabb-423e-b45d-253d08316ba3 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:50.042565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e78a54-509b-4cc0-b00d-650c635aa34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:50.043438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e78a54-509b-4cc0-b00d-650c635aa34e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:50.044219 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e78a54-509b-4cc0-b00d-650c635aa34e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:50.051530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e78a54-509b-4cc0-b00d-650c635aa34e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:50.054621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e78a54-509b-4cc0-b00d-650c635aa34e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-89e78a54-509b-4cc0-b00d-650c635aa34e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:50.063014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:50.064177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 102/193] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:39:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:50.065286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e78a54-509b-4cc0-b00d-650c635aa34e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:50.107279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 92/194] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:39:49 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:50.107950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c21ea-7f3a-44d4-97a7-7679705006da None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:50.176594 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e78a54-509b-4cc0-b00d-650c635aa34e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:50.179721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:50.212443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d331e-075a-453c-bdbb-6d4ad8c18e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:50.213193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d331e-075a-453c-bdbb-6d4ad8c18e85 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:50.213690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d331e-075a-453c-bdbb-6d4ad8c18e85 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:50.214082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d331e-075a-453c-bdbb-6d4ad8c18e85 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:50.215027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d331e-075a-453c-bdbb-6d4ad8c18e85 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:50.216232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d331e-075a-453c-bdbb-6d4ad8c18e85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:50.217741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d331e-075a-453c-bdbb-6d4ad8c18e85 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:50.227521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 103/195] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:50.232981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d331e-075a-453c-bdbb-6d4ad8c18e85 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:50.240822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:50.245548 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c21ea-7f3a-44d4-97a7-7679705006da None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:50.286830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9cfcb-9797-4850-99f6-e8f41708c7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:50.287419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9cfcb-9797-4850-99f6-e8f41708c7d3 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:50.287702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9cfcb-9797-4850-99f6-e8f41708c7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:50.287979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9cfcb-9797-4850-99f6-e8f41708c7d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:50.358910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d331e-075a-453c-bdbb-6d4ad8c18e85 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:50.364802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9cfcb-9797-4850-99f6-e8f41708c7d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:50.366296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9cfcb-9797-4850-99f6-e8f41708c7d3 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:50.411001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d028b-9b93-4bd2-8943-e0423bb0e2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:50.411676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d028b-9b93-4bd2-8943-e0423bb0e2c4 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:50.411964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d028b-9b93-4bd2-8943-e0423bb0e2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:50.412250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d028b-9b93-4bd2-8943-e0423bb0e2c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:50.413762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d028b-9b93-4bd2-8943-e0423bb0e2c4 None admin] RBAC: Adding query filter params (role.id=73a019c7fb8340a58c7dfe4550341949, scope.project.id=56fabc309a30484ebda9a05789fa5271, user.id=73a2be1f4ead4bf0a051368674237133) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:50.414136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d028b-9b93-4bd2-8943-e0423bb0e2c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:50.415681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d028b-9b93-4bd2-8943-e0423bb0e2c4 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:50.422939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 104/196] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:50 2022] GET /identity/v3/role_assignments?user.id=73a2be1f4ead4bf0a051368674237133&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=73a019c7fb8340a58c7dfe4550341949 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:50.423762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d028b-9b93-4bd2-8943-e0423bb0e2c4 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:50.474211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:50.496654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:50.501118 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 93/197] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:50.502174 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9cfcb-9797-4850-99f6-e8f41708c7d3 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:50.563423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d028b-9b93-4bd2-8943-e0423bb0e2c4 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:50.615138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49967179-c5bc-40d4-bc08-c5107bec95a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:50.615937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967179-c5bc-40d4-bc08-c5107bec95a7 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:50.616229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967179-c5bc-40d4-bc08-c5107bec95a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:50.616510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967179-c5bc-40d4-bc08-c5107bec95a7 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:50.617304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49967179-c5bc-40d4-bc08-c5107bec95a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:50.617598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49967179-c5bc-40d4-bc08-c5107bec95a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:50.619166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49967179-c5bc-40d4-bc08-c5107bec95a7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:50.623608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9cfcb-9797-4850-99f6-e8f41708c7d3 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:50.642317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 105/198] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:39:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:50.647478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49967179-c5bc-40d4-bc08-c5107bec95a7 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:50.662860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d647f-a9b6-4639-a82c-4a8fe790ae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:50.664086 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d647f-a9b6-4639-a82c-4a8fe790ae4d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:50.664572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d647f-a9b6-4639-a82c-4a8fe790ae4d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:50.665065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d647f-a9b6-4639-a82c-4a8fe790ae4d None admin] PATH_INFO: `/v3/users/9fa09b4ccdb44d84b07ad48988be75aa` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:50.666608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d647f-a9b6-4639-a82c-4a8fe790ae4d None admin] RBAC: Authorizing `identity:get_user(user_id=9fa09b4ccdb44d84b07ad48988be75aa)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:50.668441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d647f-a9b6-4639-a82c-4a8fe790ae4d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:50.671345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 94/199] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:50 2022] GET /identity/v3/users/9fa09b4ccdb44d84b07ad48988be75aa => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:50.672430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d647f-a9b6-4639-a82c-4a8fe790ae4d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:50.746367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:50.757658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:50.764705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49967179-c5bc-40d4-bc08-c5107bec95a7 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:50.782941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d647f-a9b6-4639-a82c-4a8fe790ae4d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:50.802734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c232-bef9-42d5-9c23-eb79c69e52f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:50.803995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c232-bef9-42d5-9c23-eb79c69e52f2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:50.815525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c232-bef9-42d5-9c23-eb79c69e52f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:50.815525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c232-bef9-42d5-9c23-eb79c69e52f2 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:50.815525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6c232-bef9-42d5-9c23-eb79c69e52f2 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:50.815525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6c232-bef9-42d5-9c23-eb79c69e52f2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:50.815525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 106/200] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:50 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:50.815525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c232-bef9-42d5-9c23-eb79c69e52f2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:50.827063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7ab96-98a0-4169-9933-56757a774895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:50.828245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7ab96-98a0-4169-9933-56757a774895 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:50.828907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7ab96-98a0-4169-9933-56757a774895 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:50.829393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7ab96-98a0-4169-9933-56757a774895 None admin] PATH_INFO: `/v3/users/00eb2269a55046bd9c0fa851322f177c` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:50.831749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7ab96-98a0-4169-9933-56757a774895 None admin] RBAC: Authorizing `identity:get_user(user_id=00eb2269a55046bd9c0fa851322f177c)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:50.834779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7ab96-98a0-4169-9933-56757a774895 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:50.835263 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 95/201] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:50 2022] GET /identity/v3/users/00eb2269a55046bd9c0fa851322f177c => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:50.835876 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7ab96-98a0-4169-9933-56757a774895 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:50.943744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c232-bef9-42d5-9c23-eb79c69e52f2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:50.968864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7ab96-98a0-4169-9933-56757a774895 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:50.990235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acafcde-a7a8-4b10-bbc1-79aede6e3fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:50.991406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acafcde-a7a8-4b10-bbc1-79aede6e3fed None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:50.991963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acafcde-a7a8-4b10-bbc1-79aede6e3fed None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:50.992252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acafcde-a7a8-4b10-bbc1-79aede6e3fed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:51.012093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a300007-458a-491d-bcbf-d5783551690a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:51.013440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a300007-458a-491d-bcbf-d5783551690a None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:51.014043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a300007-458a-491d-bcbf-d5783551690a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:51.014410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a300007-458a-491d-bcbf-d5783551690a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:51.015394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a300007-458a-491d-bcbf-d5783551690a None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=14afae0ea8a043bca3106a84957c35a0, user.id=00eb2269a55046bd9c0fa851322f177c) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:51.015885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a300007-458a-491d-bcbf-d5783551690a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:51.017529 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a300007-458a-491d-bcbf-d5783551690a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:51.021290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:51.025299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 96/202] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:50 2022] GET /identity/v3/role_assignments?user.id=00eb2269a55046bd9c0fa851322f177c&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=7a833c45603a4830b888e58e1232712a => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:51.026114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a300007-458a-491d-bcbf-d5783551690a None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:51.034427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:51.100787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acafcde-a7a8-4b10-bbc1-79aede6e3fed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:51.102462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acafcde-a7a8-4b10-bbc1-79aede6e3fed None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:51.222015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 107/203] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:51.228596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acafcde-a7a8-4b10-bbc1-79aede6e3fed None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:51.273441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a300007-458a-491d-bcbf-d5783551690a None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:51.301533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:51.330814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d2fed-b3ec-4775-a871-1d04404dc2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:51.331478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d2fed-b3ec-4775-a871-1d04404dc2f1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:51.331762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d2fed-b3ec-4775-a871-1d04404dc2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:51.332041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d2fed-b3ec-4775-a871-1d04404dc2f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:51.353164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acafcde-a7a8-4b10-bbc1-79aede6e3fed None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:51.425042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c80f99-63bd-44cc-b84f-128939ce59dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:51.425966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c80f99-63bd-44cc-b84f-128939ce59dc None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:51.427592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c80f99-63bd-44cc-b84f-128939ce59dc None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:51.427882 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c80f99-63bd-44cc-b84f-128939ce59dc None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:51.429094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c80f99-63bd-44cc-b84f-128939ce59dc None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:51.433845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c80f99-63bd-44cc-b84f-128939ce59dc None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:51.436689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 108/204] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:51 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:51.444153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c80f99-63bd-44cc-b84f-128939ce59dc None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:51.446865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:51.457259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d2fed-b3ec-4775-a871-1d04404dc2f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:51.461193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d2fed-b3ec-4775-a871-1d04404dc2f1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:51.617932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c80f99-63bd-44cc-b84f-128939ce59dc None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:51.646978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:51.655124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d1eb6-1fbc-4059-9d45-c63e14911496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:51.656084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d1eb6-1fbc-4059-9d45-c63e14911496 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:51.656653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d1eb6-1fbc-4059-9d45-c63e14911496 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:51.657135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d1eb6-1fbc-4059-9d45-c63e14911496 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:51.658412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d1eb6-1fbc-4059-9d45-c63e14911496 None admin] RBAC: Adding query filter params (role.id=abb740756b544900822d2326908df69e, scope.project.id=56fabc309a30484ebda9a05789fa5271, user.id=9fa09b4ccdb44d84b07ad48988be75aa) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:51.658972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d1eb6-1fbc-4059-9d45-c63e14911496 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:51.660759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d1eb6-1fbc-4059-9d45-c63e14911496 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:51.666979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 97/205] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:51.667771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 109/206] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:51 2022] GET /identity/v3/role_assignments?user.id=9fa09b4ccdb44d84b07ad48988be75aa&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=abb740756b544900822d2326908df69e => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:51.675571 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d2fed-b3ec-4775-a871-1d04404dc2f1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:51.803135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d2fed-b3ec-4775-a871-1d04404dc2f1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:51.827010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:51.840957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f6bd3-74d5-4f99-99c7-774eded69727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:51.847112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6bd3-74d5-4f99-99c7-774eded69727 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:51.847112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6bd3-74d5-4f99-99c7-774eded69727 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:51.847112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f6bd3-74d5-4f99-99c7-774eded69727 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:51.847112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f6bd3-74d5-4f99-99c7-774eded69727 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:51.847112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f6bd3-74d5-4f99-99c7-774eded69727 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:51.847112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 98/207] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:51 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:51.853043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d1eb6-1fbc-4059-9d45-c63e14911496 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:51.993098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d1eb6-1fbc-4059-9d45-c63e14911496 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:52.053604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7315bef-00d9-4f56-b36e-92a8a56b554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:52.054282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7315bef-00d9-4f56-b36e-92a8a56b554b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:52.054590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7315bef-00d9-4f56-b36e-92a8a56b554b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:52.054872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7315bef-00d9-4f56-b36e-92a8a56b554b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:52.055727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7315bef-00d9-4f56-b36e-92a8a56b554b None admin] RBAC: Adding query filter params (role.id=73a019c7fb8340a58c7dfe4550341949, scope.project.id=14afae0ea8a043bca3106a84957c35a0, user.id=00eb2269a55046bd9c0fa851322f177c) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:52.056039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7315bef-00d9-4f56-b36e-92a8a56b554b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:52.057515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7315bef-00d9-4f56-b36e-92a8a56b554b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:52.064102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 110/208] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:51 2022] GET /identity/v3/role_assignments?user.id=00eb2269a55046bd9c0fa851322f177c&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=73a019c7fb8340a58c7dfe4550341949 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:52.477857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dedec-4c30-4483-9aae-246028cf7f85 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:52.478262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dedec-4c30-4483-9aae-246028cf7f85 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:52.478454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dedec-4c30-4483-9aae-246028cf7f85 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:52.480117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 99/209] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:52.485690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bc328-7415-44a8-91f5-ce4bf4110754 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:52.485987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bc328-7415-44a8-91f5-ce4bf4110754 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:52.486269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bc328-7415-44a8-91f5-ce4bf4110754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:52.542716 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9573d-ae90-4e66-b2b4-2743e8760bb8 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:52.543024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9573d-ae90-4e66-b2b4-2743e8760bb8 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:52.543273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9573d-ae90-4e66-b2b4-2743e8760bb8 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:52.544919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 100/210] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:39:52.549579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766582-8bc3-4e0c-a477-a100e7b127d1 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:52.550257 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766582-8bc3-4e0c-a477-a100e7b127d1 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:52.550619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74766582-8bc3-4e0c-a477-a100e7b127d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:52.563195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-cc8bc328-7415-44a8-91f5-ce4bf4110754 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:52.564827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cc8bc328-7415-44a8-91f5-ce4bf4110754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:52.569503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-74766582-8bc3-4e0c-a477-a100e7b127d1 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:52.570707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-74766582-8bc3-4e0c-a477-a100e7b127d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:52.627606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:52.644855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 111/211] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:52.660391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 101/212] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:39:52.675910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bc328-7415-44a8-91f5-ce4bf4110754 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:52.685664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74766582-8bc3-4e0c-a477-a100e7b127d1 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:52.786700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74766582-8bc3-4e0c-a477-a100e7b127d1 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:52.794849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:52.801891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bc328-7415-44a8-91f5-ce4bf4110754 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:52.827574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ce604-fb67-458f-ae45-e5697bc3179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:52.828414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ce604-fb67-458f-ae45-e5697bc3179c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:52.828701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ce604-fb67-458f-ae45-e5697bc3179c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:52.828980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ce604-fb67-458f-ae45-e5697bc3179c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:52.830236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ce604-fb67-458f-ae45-e5697bc3179c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:52.831841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ce604-fb67-458f-ae45-e5697bc3179c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:52.833734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 102/213] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:52.838921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ce604-fb67-458f-ae45-e5697bc3179c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:52.848825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7948347-0588-446c-9095-50a7d509c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:52.850268 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7948347-0588-446c-9095-50a7d509c760 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:52.850765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7948347-0588-446c-9095-50a7d509c760 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:52.851727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7948347-0588-446c-9095-50a7d509c760 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:52.865411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7948347-0588-446c-9095-50a7d509c760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-f7948347-0588-446c-9095-50a7d509c760 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:52.877540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:52.878687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:52.879702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:52.887043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 112/214] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:52.888937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7948347-0588-446c-9095-50a7d509c760 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:52.950859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:52.963615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ce604-fb67-458f-ae45-e5697bc3179c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:53.007574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf599f-2c7c-43a2-ab5a-660f16ecebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:53.008260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf599f-2c7c-43a2-ab5a-660f16ecebeb None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:53.008542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf599f-2c7c-43a2-ab5a-660f16ecebeb None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:53.008824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf599f-2c7c-43a2-ab5a-660f16ecebeb None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:53.009229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7948347-0588-446c-9095-50a7d509c760 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:53.009531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf599f-2c7c-43a2-ab5a-660f16ecebeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:53.011168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf599f-2c7c-43a2-ab5a-660f16ecebeb None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:53.045006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a84363-33ae-4a3e-8597-7c5ba3c5192a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:53.045689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a84363-33ae-4a3e-8597-7c5ba3c5192a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:53.046038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a84363-33ae-4a3e-8597-7c5ba3c5192a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:53.046378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a84363-33ae-4a3e-8597-7c5ba3c5192a None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:53.047278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a84363-33ae-4a3e-8597-7c5ba3c5192a None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:53.047642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a84363-33ae-4a3e-8597-7c5ba3c5192a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:53.049094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a84363-33ae-4a3e-8597-7c5ba3c5192a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:53.057881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 113/215] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:53.063854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a84363-33ae-4a3e-8597-7c5ba3c5192a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:53.138784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 103/216] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:39:52 2022] POST /identity/v3/users => generated 297 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:39:53.181206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a84363-33ae-4a3e-8597-7c5ba3c5192a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:53.192129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:53.221382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bd798-4376-4e43-843e-138a8312a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:53.222359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bd798-4376-4e43-843e-138a8312a360 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:53.222708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bd798-4376-4e43-843e-138a8312a360 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:53.222989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bd798-4376-4e43-843e-138a8312a360 None admin] PATH_INFO: `/v3/users/b87a15e0c0054b9a8bcb1d285bfe9d1c` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:53.243243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bd798-4376-4e43-843e-138a8312a360 None admin] RBAC: Authorizing `identity:get_user(user_id=b87a15e0c0054b9a8bcb1d285bfe9d1c)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:53.245215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bd798-4376-4e43-843e-138a8312a360 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:53.248246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 114/217] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:53 2022] GET /identity/v3/users/b87a15e0c0054b9a8bcb1d285bfe9d1c => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:53.255869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf599f-2c7c-43a2-ab5a-660f16ecebeb None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:53.284317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:53.368401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf599f-2c7c-43a2-ab5a-660f16ecebeb None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:53.381096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445949f2-2a36-4027-add4-2097d887d27e None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:53.381524 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445949f2-2a36-4027-add4-2097d887d27e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:53.381880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445949f2-2a36-4027-add4-2097d887d27e None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:53.383635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 115/218] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:53.388328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836decc1-0f65-4412-b203-6dbb5bae06e3 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:53.388874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836decc1-0f65-4412-b203-6dbb5bae06e3 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:53.389147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836decc1-0f65-4412-b203-6dbb5bae06e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:53.407384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcdff9-ece8-4fa7-8837-c4c802e2d422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:53.408038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcdff9-ece8-4fa7-8837-c4c802e2d422 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:53.408319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcdff9-ece8-4fa7-8837-c4c802e2d422 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:53.408644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcdff9-ece8-4fa7-8837-c4c802e2d422 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:53.417414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-836decc1-0f65-4412-b203-6dbb5bae06e3 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:53.419183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-836decc1-0f65-4412-b203-6dbb5bae06e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:53.481767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfcdff9-ece8-4fa7-8837-c4c802e2d422 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:53.483258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfcdff9-ece8-4fa7-8837-c4c802e2d422 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:53.507698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 116/219] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:53.512577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:53.539314 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836decc1-0f65-4412-b203-6dbb5bae06e3 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:53.614981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 104/220] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:53.619282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcdff9-ece8-4fa7-8837-c4c802e2d422 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:53.670523 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:53.679628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836decc1-0f65-4412-b203-6dbb5bae06e3 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:53.732035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24dc39-875f-4567-8a7c-e002b5ab4b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:53.732883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24dc39-875f-4567-8a7c-e002b5ab4b92 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:53.733115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24dc39-875f-4567-8a7c-e002b5ab4b92 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:53.733391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24dc39-875f-4567-8a7c-e002b5ab4b92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:53.746316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b24dc39-875f-4567-8a7c-e002b5ab4b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:53.748952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcdff9-ece8-4fa7-8837-c4c802e2d422 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-6b24dc39-875f-4567-8a7c-e002b5ab4b92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:53.761472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:53.762465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:53.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 117/221] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:39:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:53.771532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24dc39-875f-4567-8a7c-e002b5ab4b92 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:53.779291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:53.807283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a8975-b246-4b83-ad67-d3c27ff24bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:53.809365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a8975-b246-4b83-ad67-d3c27ff24bff None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:53.811184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a8975-b246-4b83-ad67-d3c27ff24bff None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:53.811932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a8975-b246-4b83-ad67-d3c27ff24bff None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:53.814826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a8975-b246-4b83-ad67-d3c27ff24bff None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:53.817479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a8975-b246-4b83-ad67-d3c27ff24bff None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:53.822280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 105/222] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:53 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:53.829288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a8975-b246-4b83-ad67-d3c27ff24bff None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:53.939746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24dc39-875f-4567-8a7c-e002b5ab4b92 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:53.986282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935eb835-0fd2-471a-85ea-5245b4105305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:53.986999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eb835-0fd2-471a-85ea-5245b4105305 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:53.987284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eb835-0fd2-471a-85ea-5245b4105305 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:53.987565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eb835-0fd2-471a-85ea-5245b4105305 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:53.988330 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935eb835-0fd2-471a-85ea-5245b4105305 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:53.988642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935eb835-0fd2-471a-85ea-5245b4105305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:53.990048 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935eb835-0fd2-471a-85ea-5245b4105305 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:54.012645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 118/223] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:54.019717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935eb835-0fd2-471a-85ea-5245b4105305 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:54.055816 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:54.127008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a8975-b246-4b83-ad67-d3c27ff24bff None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:54.156993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935eb835-0fd2-471a-85ea-5245b4105305 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:54.179635 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403d420-fee8-4306-9d4a-562f39a96453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:54.180232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403d420-fee8-4306-9d4a-562f39a96453 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:54.180512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403d420-fee8-4306-9d4a-562f39a96453 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:54.180786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403d420-fee8-4306-9d4a-562f39a96453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:54.181622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403d420-fee8-4306-9d4a-562f39a96453 None admin] RBAC: Adding query filter params (role.id=5dce8d92cffa40f79dfa1c03724c47e2, scope.project.id=14afae0ea8a043bca3106a84957c35a0, user.id=b87a15e0c0054b9a8bcb1d285bfe9d1c) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:54.181938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403d420-fee8-4306-9d4a-562f39a96453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:54.185338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403d420-fee8-4306-9d4a-562f39a96453 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:54.192317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 106/224] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:53 2022] GET /identity/v3/role_assignments?user.id=b87a15e0c0054b9a8bcb1d285bfe9d1c&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 259 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:54.193598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a6f4c-adf8-40b3-b888-4e096577752b None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:54.193883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a6f4c-adf8-40b3-b888-4e096577752b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:54.194159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a6f4c-adf8-40b3-b888-4e096577752b None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:54.200819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 107/225] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:54.203543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82870684-97b8-4a2e-b3e4-4bcb23cbf744 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:54.203890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82870684-97b8-4a2e-b3e4-4bcb23cbf744 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:54.204177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82870684-97b8-4a2e-b3e4-4bcb23cbf744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:54.214852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd23b17-78d9-4160-b9f0-c94d7fb7f416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:54.216560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23b17-78d9-4160-b9f0-c94d7fb7f416 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:54.217013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23b17-78d9-4160-b9f0-c94d7fb7f416 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:54.217431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23b17-78d9-4160-b9f0-c94d7fb7f416 None admin] PATH_INFO: `/v3/users/00eb2269a55046bd9c0fa851322f177c` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:54.218991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd23b17-78d9-4160-b9f0-c94d7fb7f416 None admin] RBAC: Authorizing `identity:get_user(user_id=00eb2269a55046bd9c0fa851322f177c)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:54.220624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd23b17-78d9-4160-b9f0-c94d7fb7f416 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:54.222448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 119/226] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:54 2022] GET /identity/v3/users/00eb2269a55046bd9c0fa851322f177c => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:54.227544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-82870684-97b8-4a2e-b3e4-4bcb23cbf744 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:54.229489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-82870684-97b8-4a2e-b3e4-4bcb23cbf744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:54.230524 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd23b17-78d9-4160-b9f0-c94d7fb7f416 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:54.312249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:54.351678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 108/227] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:54.384416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:54.395187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82870684-97b8-4a2e-b3e4-4bcb23cbf744 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:54.412156 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd23b17-78d9-4160-b9f0-c94d7fb7f416 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:54.452303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84277b4-1d17-4bfb-83ab-7ee88dca9881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:54.453635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84277b4-1d17-4bfb-83ab-7ee88dca9881 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:54.454917 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84277b4-1d17-4bfb-83ab-7ee88dca9881 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:54.455208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84277b4-1d17-4bfb-83ab-7ee88dca9881 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:54.457077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84277b4-1d17-4bfb-83ab-7ee88dca9881 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:54.458656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84277b4-1d17-4bfb-83ab-7ee88dca9881 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:54.461040 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 120/228] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:54 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:54.467576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84277b4-1d17-4bfb-83ab-7ee88dca9881 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:54.542515 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82870684-97b8-4a2e-b3e4-4bcb23cbf744 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:54.578978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31342e-9f3e-4175-9c80-b74b014d4b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:54.579883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31342e-9f3e-4175-9c80-b74b014d4b88 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:54.580169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31342e-9f3e-4175-9c80-b74b014d4b88 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:54.580442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31342e-9f3e-4175-9c80-b74b014d4b88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:54.591378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31342e-9f3e-4175-9c80-b74b014d4b88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-5c31342e-9f3e-4175-9c80-b74b014d4b88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:54.603142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:54.604104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 109/229] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:39:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:54.607440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31342e-9f3e-4175-9c80-b74b014d4b88 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:54.661442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:54.676253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84277b4-1d17-4bfb-83ab-7ee88dca9881 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:54.720055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31342e-9f3e-4175-9c80-b74b014d4b88 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:54.755171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc3126-5b93-495b-b576-87ca41d7c171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:54.755941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84915a-f714-4a02-8028-67a13d1f4e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:54.756082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc3126-5b93-495b-b576-87ca41d7c171 None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:54.756355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc3126-5b93-495b-b576-87ca41d7c171 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:54.756611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84915a-f714-4a02-8028-67a13d1f4e73 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:54.756712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc3126-5b93-495b-b576-87ca41d7c171 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0/users/00eb2269a55046bd9c0fa851322f177c/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:54.756891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84915a-f714-4a02-8028-67a13d1f4e73 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:54.757167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84915a-f714-4a02-8028-67a13d1f4e73 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:54.757956 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84915a-f714-4a02-8028-67a13d1f4e73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:54.758207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84915a-f714-4a02-8028-67a13d1f4e73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:54.758694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc3126-5b93-495b-b576-87ca41d7c171 None admin] RBAC: Authorizing `identity:create_grant(project_id=14afae0ea8a043bca3106a84957c35a0, user_id=00eb2269a55046bd9c0fa851322f177c, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:54.759613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84915a-f714-4a02-8028-67a13d1f4e73 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:54.760138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc3126-5b93-495b-b576-87ca41d7c171 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:54.768622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 110/230] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:39:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:54.774175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84915a-f714-4a02-8028-67a13d1f4e73 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:54.878128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84915a-f714-4a02-8028-67a13d1f4e73 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:54.880916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 121/231] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:39:54 2022] PUT /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0/users/00eb2269a55046bd9c0fa851322f177c/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:54.886947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:54.915796 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfac4e-5d64-4188-8db9-e55bb0780f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:54.916694 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfac4e-5d64-4188-8db9-e55bb0780f21 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:54.916976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfac4e-5d64-4188-8db9-e55bb0780f21 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:54.917249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfac4e-5d64-4188-8db9-e55bb0780f21 None admin] PATH_INFO: `/v3/users/00eb2269a55046bd9c0fa851322f177c` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:54.919021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfac4e-5d64-4188-8db9-e55bb0780f21 None admin] RBAC: Authorizing `identity:get_user(user_id=00eb2269a55046bd9c0fa851322f177c)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:54.920471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfac4e-5d64-4188-8db9-e55bb0780f21 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:54.922457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 111/232] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:54 2022] GET /identity/v3/users/00eb2269a55046bd9c0fa851322f177c => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:54.928140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc3126-5b93-495b-b576-87ca41d7c171 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:54.951228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:55.067842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc3126-5b93-495b-b576-87ca41d7c171 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:55.104858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21edce-1529-4068-96e3-2b78b471f74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:55.105730 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21edce-1529-4068-96e3-2b78b471f74d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:55.106098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21edce-1529-4068-96e3-2b78b471f74d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:55.106403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21edce-1529-4068-96e3-2b78b471f74d None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:55.107610 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21edce-1529-4068-96e3-2b78b471f74d None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:55.109061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21edce-1529-4068-96e3-2b78b471f74d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:55.110799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 122/233] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:54 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:55.116734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfac4e-5d64-4188-8db9-e55bb0780f21 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:55.220321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfac4e-5d64-4188-8db9-e55bb0780f21 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:55.254330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b16178-5449-4c0b-bf41-8808c30d492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:55.255264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16178-5449-4c0b-bf41-8808c30d492f None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:55.255544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16178-5449-4c0b-bf41-8808c30d492f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:55.255816 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b16178-5449-4c0b-bf41-8808c30d492f None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0/users/00eb2269a55046bd9c0fa851322f177c/roles/73a019c7fb8340a58c7dfe4550341949` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:55.258028 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b16178-5449-4c0b-bf41-8808c30d492f None admin] RBAC: Authorizing `identity:create_grant(project_id=14afae0ea8a043bca3106a84957c35a0, user_id=00eb2269a55046bd9c0fa851322f177c, role_id=73a019c7fb8340a58c7dfe4550341949)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:55.259496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b16178-5449-4c0b-bf41-8808c30d492f None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:55.392425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 112/234] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:39:55 2022] PUT /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0/users/00eb2269a55046bd9c0fa851322f177c/roles/73a019c7fb8340a58c7dfe4550341949 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:55.694139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295f18b-ca82-4d2d-83a0-2e19543b8aba None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:55.694139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295f18b-ca82-4d2d-83a0-2e19543b8aba None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:55.694139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295f18b-ca82-4d2d-83a0-2e19543b8aba None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:55.694139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 123/235] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:55.700665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04411dc0-177e-4136-8ede-c98328f23a68 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:55.701190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04411dc0-177e-4136-8ede-c98328f23a68 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:55.701677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04411dc0-177e-4136-8ede-c98328f23a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:55.718774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-04411dc0-177e-4136-8ede-c98328f23a68 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:55.720298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-04411dc0-177e-4136-8ede-c98328f23a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:55.752124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:55.795651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 113/236] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:55.835449 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295f18b-ca82-4d2d-83a0-2e19543b8aba None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:55.970431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:55.988127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e668d-ac89-45e8-ba7b-454edbd806c5 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:55.988464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e668d-ac89-45e8-ba7b-454edbd806c5 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:55.988686 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e668d-ac89-45e8-ba7b-454edbd806c5 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:55.990082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 114/237] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:55.993971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295f18b-ca82-4d2d-83a0-2e19543b8aba None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:55.999818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdaeff-030f-4e0f-9c15-8ac2a9bf616c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:56.000102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdaeff-030f-4e0f-9c15-8ac2a9bf616c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:56.000382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdaeff-030f-4e0f-9c15-8ac2a9bf616c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:56.032870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba013b70-f640-49e8-98cd-19ebe4331d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:56.033541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba013b70-f640-49e8-98cd-19ebe4331d0f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:56.034881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba013b70-f640-49e8-98cd-19ebe4331d0f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:56.034881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba013b70-f640-49e8-98cd-19ebe4331d0f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:56.042422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-81fdaeff-030f-4e0f-9c15-8ac2a9bf616c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:56.044403 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-81fdaeff-030f-4e0f-9c15-8ac2a9bf616c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:56.046499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba013b70-f640-49e8-98cd-19ebe4331d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-ba013b70-f640-49e8-98cd-19ebe4331d0f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:56.055279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:56.056196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:39:56.057107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:56.061879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 124/238] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:56.063129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba013b70-f640-49e8-98cd-19ebe4331d0f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:56.185448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba013b70-f640-49e8-98cd-19ebe4331d0f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:56.229146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08041812-f850-4ad6-8adb-4da9fb80552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:56.229802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08041812-f850-4ad6-8adb-4da9fb80552d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:56.230097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08041812-f850-4ad6-8adb-4da9fb80552d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:56.230778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08041812-f850-4ad6-8adb-4da9fb80552d None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:56.231709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08041812-f850-4ad6-8adb-4da9fb80552d None admin] RBAC: Adding query filter params (name=reader) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:56.232028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08041812-f850-4ad6-8adb-4da9fb80552d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:56.233448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08041812-f850-4ad6-8adb-4da9fb80552d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:56.242310 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 115/239] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:56.244173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 125/240] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:56.249822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdaeff-030f-4e0f-9c15-8ac2a9bf616c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:56.283427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08041812-f850-4ad6-8adb-4da9fb80552d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:56.288493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:56.394885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:56.397640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdaeff-030f-4e0f-9c15-8ac2a9bf616c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:56.425153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08041812-f850-4ad6-8adb-4da9fb80552d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:56.432281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaa77e-1cc3-4d05-8d03-491bdd2045c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:56.433172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaa77e-1cc3-4d05-8d03-491bdd2045c0 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:56.433455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaa77e-1cc3-4d05-8d03-491bdd2045c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:56.433731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaa77e-1cc3-4d05-8d03-491bdd2045c0 None admin] PATH_INFO: `/v3/users/1b502b08b2544366beffe285fa1830f4` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:56.450318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaa77e-1cc3-4d05-8d03-491bdd2045c0 None admin] RBAC: Authorizing `identity:get_user(user_id=1b502b08b2544366beffe285fa1830f4)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:56.451828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaa77e-1cc3-4d05-8d03-491bdd2045c0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:56.453885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 116/241] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:56 2022] GET /identity/v3/users/1b502b08b2544366beffe285fa1830f4 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:56.467681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaa77e-1cc3-4d05-8d03-491bdd2045c0 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:56.479259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420471a-968e-49e4-a8a9-785e324e42e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:56.480012 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420471a-968e-49e4-a8a9-785e324e42e8 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:56.480304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420471a-968e-49e4-a8a9-785e324e42e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:56.480583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420471a-968e-49e4-a8a9-785e324e42e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:56.481995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420471a-968e-49e4-a8a9-785e324e42e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:56.483498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420471a-968e-49e4-a8a9-785e324e42e8 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:56.487484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 126/242] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:39:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:56.503259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420471a-968e-49e4-a8a9-785e324e42e8 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:56.578719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaa77e-1cc3-4d05-8d03-491bdd2045c0 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:56.616008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:56.627244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c01e6-51e6-4f76-80ad-8272272cf970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:56.627984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c01e6-51e6-4f76-80ad-8272272cf970 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:56.628328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c01e6-51e6-4f76-80ad-8272272cf970 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:56.628666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c01e6-51e6-4f76-80ad-8272272cf970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:56.649574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420471a-968e-49e4-a8a9-785e324e42e8 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:56.651538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:56.685414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e76abc-caf2-4231-a7bc-a75b6f22ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:56.686294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e76abc-caf2-4231-a7bc-a75b6f22ea4f None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:56.687109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e76abc-caf2-4231-a7bc-a75b6f22ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:56.687741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e76abc-caf2-4231-a7bc-a75b6f22ea4f None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:56.689128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e76abc-caf2-4231-a7bc-a75b6f22ea4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:56.691299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e76abc-caf2-4231-a7bc-a75b6f22ea4f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:56.700491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c01e6-51e6-4f76-80ad-8272272cf970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:56.702133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c01e6-51e6-4f76-80ad-8272272cf970 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:56.786731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 127/243] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:39:56 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:39:56.788869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fbf60-c2ba-4b37-bb43-b1f4b0933586 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:56.789360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fbf60-c2ba-4b37-bb43-b1f4b0933586 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:56.789867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fbf60-c2ba-4b37-bb43-b1f4b0933586 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:56.799185 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 128/244] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:56.805164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 117/245] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:39:56.807554 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432b35a-3949-4e28-9eb3-2b5e1521b31b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:56.807842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432b35a-3949-4e28-9eb3-2b5e1521b31b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:56.808130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432b35a-3949-4e28-9eb3-2b5e1521b31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:56.811318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c01e6-51e6-4f76-80ad-8272272cf970 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:56.847263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e432b35a-3949-4e28-9eb3-2b5e1521b31b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:56.848508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e432b35a-3949-4e28-9eb3-2b5e1521b31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:56.935722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:56.979431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 129/246] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:39:56.992381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c01e6-51e6-4f76-80ad-8272272cf970 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:57.008940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432b35a-3949-4e28-9eb3-2b5e1521b31b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:57.033002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8947e-962b-4cab-aff8-e43542ac62c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:57.033002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8947e-962b-4cab-aff8-e43542ac62c2 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:57.033002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8947e-962b-4cab-aff8-e43542ac62c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:57.033350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8947e-962b-4cab-aff8-e43542ac62c2 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:57.034430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8947e-962b-4cab-aff8-e43542ac62c2 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:57.036120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8947e-962b-4cab-aff8-e43542ac62c2 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:57.038628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 118/247] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:56 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:57.044238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8947e-962b-4cab-aff8-e43542ac62c2 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:57.087001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:57.144335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432b35a-3949-4e28-9eb3-2b5e1521b31b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:57.165617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8947e-962b-4cab-aff8-e43542ac62c2 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:57.193028 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:57.195593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b13a3-94fa-43de-ab67-97b4cc2b209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:57.196405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b13a3-94fa-43de-ab67-97b4cc2b209d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:57.196688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b13a3-94fa-43de-ab67-97b4cc2b209d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:57.196963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b13a3-94fa-43de-ab67-97b4cc2b209d None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:57.216331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b13a3-94fa-43de-ab67-97b4cc2b209d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:57.219012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf9120-6bbf-4838-af65-6f875ce3dddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:57.219678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9120-6bbf-4838-af65-6f875ce3dddd None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:57.219962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9120-6bbf-4838-af65-6f875ce3dddd None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:57.220239 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9120-6bbf-4838-af65-6f875ce3dddd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:57.221076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf9120-6bbf-4838-af65-6f875ce3dddd None admin] RBAC: Adding query filter params (role.id=abb740756b544900822d2326908df69e, scope.project.id=14afae0ea8a043bca3106a84957c35a0, user.id=1b502b08b2544366beffe285fa1830f4) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:57.221389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf9120-6bbf-4838-af65-6f875ce3dddd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:57.222918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf9120-6bbf-4838-af65-6f875ce3dddd None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-254b13a3-94fa-43de-ab67-97b4cc2b209d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:57.226939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:57.227806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:39:57.228643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:57.232683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 130/248] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:39:57.236502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 119/249] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:57 2022] GET /identity/v3/role_assignments?user.id=1b502b08b2544366beffe285fa1830f4&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=abb740756b544900822d2326908df69e => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:57.237839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b13a3-94fa-43de-ab67-97b4cc2b209d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:57.359811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b13a3-94fa-43de-ab67-97b4cc2b209d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:57.379552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:57.402144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c810-c555-46de-abfc-cd73579020df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:57.402903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c810-c555-46de-abfc-cd73579020df None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:57.403188 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c810-c555-46de-abfc-cd73579020df None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:57.403463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c810-c555-46de-abfc-cd73579020df None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:57.404244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c810-c555-46de-abfc-cd73579020df None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:57.404554 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c810-c555-46de-abfc-cd73579020df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:57.406008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c810-c555-46de-abfc-cd73579020df None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:57.416349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 131/250] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:39:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:57.422413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf9120-6bbf-4838-af65-6f875ce3dddd None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:57.551728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf9120-6bbf-4838-af65-6f875ce3dddd None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:57.556570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c9cce-9c29-4697-b646-a28818a7d0b9 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:57.556857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c9cce-9c29-4697-b646-a28818a7d0b9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:57.557133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c9cce-9c29-4697-b646-a28818a7d0b9 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:57.562953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 132/251] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:39:57.564110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e6636-6e7c-460e-ad99-c2b90f87ef1a None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:57.564402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e6636-6e7c-460e-ad99-c2b90f87ef1a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:57.564678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e6636-6e7c-460e-ad99-c2b90f87ef1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:57.584669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d85e6636-6e7c-460e-ad99-c2b90f87ef1a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:57.585964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d85e6636-6e7c-460e-ad99-c2b90f87ef1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:57.604609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c62096-b70b-48a1-92fb-8e9a1c25bd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:57.605431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c62096-b70b-48a1-92fb-8e9a1c25bd9a None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:57.605745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c62096-b70b-48a1-92fb-8e9a1c25bd9a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:57.606019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c62096-b70b-48a1-92fb-8e9a1c25bd9a None admin] PATH_INFO: `/v3/users/b87a15e0c0054b9a8bcb1d285bfe9d1c` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:57.611323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c62096-b70b-48a1-92fb-8e9a1c25bd9a None admin] RBAC: Authorizing `identity:get_user(user_id=b87a15e0c0054b9a8bcb1d285bfe9d1c)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:57.612710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c62096-b70b-48a1-92fb-8e9a1c25bd9a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:57.614735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 120/252] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:57 2022] GET /identity/v3/users/b87a15e0c0054b9a8bcb1d285bfe9d1c => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:57.620194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c62096-b70b-48a1-92fb-8e9a1c25bd9a None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:57.659253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 133/253] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 25 05:39:57.684588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e6636-6e7c-460e-ad99-c2b90f87ef1a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:57.759211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:57.793797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e6636-6e7c-460e-ad99-c2b90f87ef1a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:57.806285 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:57.831611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5995a-0111-40e3-a8e9-70aa8d7a2773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:57.832761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5995a-0111-40e3-a8e9-70aa8d7a2773 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:57.833216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5995a-0111-40e3-a8e9-70aa8d7a2773 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:57.833634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5995a-0111-40e3-a8e9-70aa8d7a2773 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:57.848145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5995a-0111-40e3-a8e9-70aa8d7a2773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-c4b5995a-0111-40e3-a8e9-70aa8d7a2773 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:57.856444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:57.857525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:39:57.858806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 134/254] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:39:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:57.872905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c62096-b70b-48a1-92fb-8e9a1c25bd9a None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:57.874307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5995a-0111-40e3-a8e9-70aa8d7a2773 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:57.946133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808f63d-75a3-41bf-b953-3f9286fefcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:57.947043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808f63d-75a3-41bf-b953-3f9286fefcff None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:57.947328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808f63d-75a3-41bf-b953-3f9286fefcff None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:57.947605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808f63d-75a3-41bf-b953-3f9286fefcff None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:57.949463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808f63d-75a3-41bf-b953-3f9286fefcff None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:57.950954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808f63d-75a3-41bf-b953-3f9286fefcff None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:57.952692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 121/255] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:57 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:57.959228 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808f63d-75a3-41bf-b953-3f9286fefcff None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:58.078440 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5995a-0111-40e3-a8e9-70aa8d7a2773 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:58.109541 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808f63d-75a3-41bf-b953-3f9286fefcff None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:58.120302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c613c4-0ecb-437d-ad43-c6209c6199f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:58.121090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c613c4-0ecb-437d-ad43-c6209c6199f9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.121413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c613c4-0ecb-437d-ad43-c6209c6199f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.121698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c613c4-0ecb-437d-ad43-c6209c6199f9 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.122465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c613c4-0ecb-437d-ad43-c6209c6199f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:58.122811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c613c4-0ecb-437d-ad43-c6209c6199f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:58.124288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c613c4-0ecb-437d-ad43-c6209c6199f9 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:58.137581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 135/256] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:39:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:58.137756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:58.137842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3c5c-0102-4030-a11a-b47115c596f7 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.137842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3c5c-0102-4030-a11a-b47115c596f7 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.137842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3c5c-0102-4030-a11a-b47115c596f7 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.141249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 136/257] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:39:58.142063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe3c5c-0102-4030-a11a-b47115c596f7 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:58.153507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcf899-ed75-404f-b65d-bba27f4b8a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:58.154827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcf899-ed75-404f-b65d-bba27f4b8a99 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.155995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcf899-ed75-404f-b65d-bba27f4b8a99 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.155995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcf899-ed75-404f-b65d-bba27f4b8a99 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0/users/b87a15e0c0054b9a8bcb1d285bfe9d1c/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.157999 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcf899-ed75-404f-b65d-bba27f4b8a99 None admin] RBAC: Authorizing `identity:create_grant(project_id=14afae0ea8a043bca3106a84957c35a0, user_id=b87a15e0c0054b9a8bcb1d285bfe9d1c, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:58.159898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcf899-ed75-404f-b65d-bba27f4b8a99 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:58.176387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:58.244294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 122/258] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:39:57 2022] PUT /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0/users/b87a15e0c0054b9a8bcb1d285bfe9d1c/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:58.246013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6a7c2-7a31-48be-ae5e-be5a479d75d0 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.246576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6a7c2-7a31-48be-ae5e-be5a479d75d0 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.247148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6a7c2-7a31-48be-ae5e-be5a479d75d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.253720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe3c5c-0102-4030-a11a-b47115c596f7 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:58.288114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-72c6a7c2-7a31-48be-ae5e-be5a479d75d0 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:58.289842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-72c6a7c2-7a31-48be-ae5e-be5a479d75d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:58.295320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f98192-2bc8-4909-b786-5eec603d4655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:58.295676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f98192-2bc8-4909-b786-5eec603d4655 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.295980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f98192-2bc8-4909-b786-5eec603d4655 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.296292 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f98192-2bc8-4909-b786-5eec603d4655 None admin] PATH_INFO: `/v3/users/00eb2269a55046bd9c0fa851322f177c` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.302571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f98192-2bc8-4909-b786-5eec603d4655 None admin] RBAC: Authorizing `identity:get_user(user_id=00eb2269a55046bd9c0fa851322f177c)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:58.304469 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f98192-2bc8-4909-b786-5eec603d4655 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:58.307230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 137/259] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:58 2022] GET /identity/v3/users/00eb2269a55046bd9c0fa851322f177c => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:58.315230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f98192-2bc8-4909-b786-5eec603d4655 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:58.438055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 123/260] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:39:58.490580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f98192-2bc8-4909-b786-5eec603d4655 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:58.494905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:58.523521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6a7c2-7a31-48be-ae5e-be5a479d75d0 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:58.542496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56df262-75c2-4d19-89e3-6049a1fb6cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:58.542496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df262-75c2-4d19-89e3-6049a1fb6cf1 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.542496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df262-75c2-4d19-89e3-6049a1fb6cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.542496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df262-75c2-4d19-89e3-6049a1fb6cf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.610634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56df262-75c2-4d19-89e3-6049a1fb6cf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:58.612118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56df262-75c2-4d19-89e3-6049a1fb6cf1 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:58.660596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6a7c2-7a31-48be-ae5e-be5a479d75d0 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:58.680122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:58.699101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9fb3f-ae75-4cff-9338-698b22d02d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:58.700252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9fb3f-ae75-4cff-9338-698b22d02d06 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.700754 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9fb3f-ae75-4cff-9338-698b22d02d06 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.701813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9fb3f-ae75-4cff-9338-698b22d02d06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.713441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9fb3f-ae75-4cff-9338-698b22d02d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:39:58.715247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:58.722498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 138/261] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:39:58.724836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56df262-75c2-4d19-89e3-6049a1fb6cf1 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-94d9fb3f-ae75-4cff-9338-698b22d02d06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:39:58.732938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:39:58.733948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:39:58.734829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 124/262] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:39:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:39:58.764099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9fb3f-ae75-4cff-9338-698b22d02d06 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:58.948900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56df262-75c2-4d19-89e3-6049a1fb6cf1 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:58.988912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9fb3f-ae75-4cff-9338-698b22d02d06 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:59.002603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907dc45-a4a6-46b3-ab5f-75e3110d92a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:59.003890 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907dc45-a4a6-46b3-ab5f-75e3110d92a0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.004380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907dc45-a4a6-46b3-ab5f-75e3110d92a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.004855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907dc45-a4a6-46b3-ab5f-75e3110d92a0 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.009251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907dc45-a4a6-46b3-ab5f-75e3110d92a0 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:59.011265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907dc45-a4a6-46b3-ab5f-75e3110d92a0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:59.015909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 139/263] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:58 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:59.024217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907dc45-a4a6-46b3-ab5f-75e3110d92a0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:59.067244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e08a0f-4b5d-40e0-89c4-b19f03b697c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:59.068301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e08a0f-4b5d-40e0-89c4-b19f03b697c1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.069833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e08a0f-4b5d-40e0-89c4-b19f03b697c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.070317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e08a0f-4b5d-40e0-89c4-b19f03b697c1 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.071460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e08a0f-4b5d-40e0-89c4-b19f03b697c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:59.073039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e08a0f-4b5d-40e0-89c4-b19f03b697c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:59.074730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e08a0f-4b5d-40e0-89c4-b19f03b697c1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:59.085829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 125/264] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:39:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:59.092055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e08a0f-4b5d-40e0-89c4-b19f03b697c1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:59.150836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:59.177044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:59.222525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907dc45-a4a6-46b3-ab5f-75e3110d92a0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:59.291731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e08a0f-4b5d-40e0-89c4-b19f03b697c1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:59.310502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff2d1b-210c-4758-8aab-5bd042bc2b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:59.311166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff2d1b-210c-4758-8aab-5bd042bc2b96 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.311446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff2d1b-210c-4758-8aab-5bd042bc2b96 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.311717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff2d1b-210c-4758-8aab-5bd042bc2b96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.312606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff2d1b-210c-4758-8aab-5bd042bc2b96 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=14afae0ea8a043bca3106a84957c35a0, user.id=00eb2269a55046bd9c0fa851322f177c) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:59.312966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff2d1b-210c-4758-8aab-5bd042bc2b96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:59.314425 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff2d1b-210c-4758-8aab-5bd042bc2b96 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:59.330384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 140/265] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:59 2022] GET /identity/v3/role_assignments?user.id=00eb2269a55046bd9c0fa851322f177c&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=7a833c45603a4830b888e58e1232712a => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:59.364702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5da2-e301-4656-a801-0465cfeb1d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:59.365978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5da2-e301-4656-a801-0465cfeb1d82 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.366459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5da2-e301-4656-a801-0465cfeb1d82 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.366961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a5da2-e301-4656-a801-0465cfeb1d82 None admin] PATH_INFO: `/v3/users/00eb2269a55046bd9c0fa851322f177c` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.368355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a5da2-e301-4656-a801-0465cfeb1d82 None admin] RBAC: Authorizing `identity:get_user(user_id=00eb2269a55046bd9c0fa851322f177c)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:59.370020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a5da2-e301-4656-a801-0465cfeb1d82 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:59.372070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 126/266] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:39:59 2022] GET /identity/v3/users/00eb2269a55046bd9c0fa851322f177c => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:59.377935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff2d1b-210c-4758-8aab-5bd042bc2b96 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:59.527346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff2d1b-210c-4758-8aab-5bd042bc2b96 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:59.564635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:59.570247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db04817-4e98-4fc3-a074-991503a5fb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:59.570993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db04817-4e98-4fc3-a074-991503a5fb4a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.571275 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db04817-4e98-4fc3-a074-991503a5fb4a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.571555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db04817-4e98-4fc3-a074-991503a5fb4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.651014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db04817-4e98-4fc3-a074-991503a5fb4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:59.652523 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db04817-4e98-4fc3-a074-991503a5fb4a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:59.763968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 141/267] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:39:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:39:59.769696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5da2-e301-4656-a801-0465cfeb1d82 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:59.877302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:59.880243 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a5da2-e301-4656-a801-0465cfeb1d82 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:59.937023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37877e1a-5002-4a0e-b5ec-4a3ec656b4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:59.937893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37877e1a-5002-4a0e-b5ec-4a3ec656b4d6 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.938171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37877e1a-5002-4a0e-b5ec-4a3ec656b4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.938439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37877e1a-5002-4a0e-b5ec-4a3ec656b4d6 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.939688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37877e1a-5002-4a0e-b5ec-4a3ec656b4d6 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:59.941127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37877e1a-5002-4a0e-b5ec-4a3ec656b4d6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:59.943170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 127/268] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:39:59 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:59.948946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db04817-4e98-4fc3-a074-991503a5fb4a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:59.987542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:00.066789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db04817-4e98-4fc3-a074-991503a5fb4a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:00.101796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea97b46-1345-433b-aebb-995cee306b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:00.102491 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97b46-1345-433b-aebb-995cee306b6c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.102767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97b46-1345-433b-aebb-995cee306b6c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.103039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97b46-1345-433b-aebb-995cee306b6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.103881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea97b46-1345-433b-aebb-995cee306b6c None admin] RBAC: Adding query filter params (role.id=73a019c7fb8340a58c7dfe4550341949, scope.project.id=14afae0ea8a043bca3106a84957c35a0, user.id=00eb2269a55046bd9c0fa851322f177c) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:00.104190 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea97b46-1345-433b-aebb-995cee306b6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:00.106183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea97b46-1345-433b-aebb-995cee306b6c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:00.113055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 142/269] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:39:59 2022] GET /identity/v3/role_assignments?user.id=00eb2269a55046bd9c0fa851322f177c&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=73a019c7fb8340a58c7dfe4550341949 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:00.181933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce0cf4-4d19-4727-858d-d9a3b0ee8b46 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.182611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce0cf4-4d19-4727-858d-d9a3b0ee8b46 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.183104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce0cf4-4d19-4727-858d-d9a3b0ee8b46 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.185658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 128/270] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:00.191711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faca50-b9fe-4f4a-9e75-357d8ce59898 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.192683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faca50-b9fe-4f4a-9e75-357d8ce59898 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.193170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faca50-b9fe-4f4a-9e75-357d8ce59898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.211599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-29faca50-b9fe-4f4a-9e75-357d8ce59898 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:00.213186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-29faca50-b9fe-4f4a-9e75-357d8ce59898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:00.310570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 143/271] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:00.334497 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce0cf4-4d19-4727-858d-d9a3b0ee8b46 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:00.337549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c3c6-bb7e-4cc2-89c0-47ac08a35d1b None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.337839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c3c6-bb7e-4cc2-89c0-47ac08a35d1b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.338115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c3c6-bb7e-4cc2-89c0-47ac08a35d1b None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.339670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 144/272] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:00.345228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc327e9-677c-470c-8c19-a58ecc21efeb None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.345519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc327e9-677c-470c-8c19-a58ecc21efeb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.345798 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc327e9-677c-470c-8c19-a58ecc21efeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.365436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-dbc327e9-677c-470c-8c19-a58ecc21efeb None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:00.367350 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-dbc327e9-677c-470c-8c19-a58ecc21efeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:00.394574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:00.455208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 145/273] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:00.472916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce0cf4-4d19-4727-858d-d9a3b0ee8b46 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:00.485944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc327e9-677c-470c-8c19-a58ecc21efeb None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:00.520622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3c6ae-a231-4506-b2fc-fa39d1eab13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:00.521426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3c6ae-a231-4506-b2fc-fa39d1eab13b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.521705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3c6ae-a231-4506-b2fc-fa39d1eab13b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.521978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3c6ae-a231-4506-b2fc-fa39d1eab13b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.523306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3c6ae-a231-4506-b2fc-fa39d1eab13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:00.525455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3c6ae-a231-4506-b2fc-fa39d1eab13b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:00.529066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 129/274] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:00.534032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3c6ae-a231-4506-b2fc-fa39d1eab13b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:00.578683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:00.624192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc327e9-677c-470c-8c19-a58ecc21efeb None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:00.651237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3c6ae-a231-4506-b2fc-fa39d1eab13b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:00.661430 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba58b09-3ebf-413a-91df-52a120294b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:00.662234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba58b09-3ebf-413a-91df-52a120294b7d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.662538 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba58b09-3ebf-413a-91df-52a120294b7d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.663076 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba58b09-3ebf-413a-91df-52a120294b7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.673973 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba58b09-3ebf-413a-91df-52a120294b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-eba58b09-3ebf-413a-91df-52a120294b7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:00.681754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:00.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:00.683511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 146/275] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:00.684994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628423c-aee2-4bf7-a0db-5666193d6674 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.684994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628423c-aee2-4bf7-a0db-5666193d6674 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.684994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628423c-aee2-4bf7-a0db-5666193d6674 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.686240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 147/276] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:00.691310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ac933-71b1-4790-ae13-67a1a8f79a0b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.691603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ac933-71b1-4790-ae13-67a1a8f79a0b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.691704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebce07-aba4-4cc6-82f6-32e01b1d2410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:00.691892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ac933-71b1-4790-ae13-67a1a8f79a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.692312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebce07-aba4-4cc6-82f6-32e01b1d2410 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.692604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebce07-aba4-4cc6-82f6-32e01b1d2410 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.692886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebce07-aba4-4cc6-82f6-32e01b1d2410 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.693609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebce07-aba4-4cc6-82f6-32e01b1d2410 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:00.695149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebce07-aba4-4cc6-82f6-32e01b1d2410 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:00.709221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f50ac933-71b1-4790-ae13-67a1a8f79a0b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:00.710403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f50ac933-71b1-4790-ae13-67a1a8f79a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:00.721836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:00.776269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 148/277] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:00.786628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ac933-71b1-4790-ae13-67a1a8f79a0b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:00.803824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 130/278] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:00 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:00.804683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebce07-aba4-4cc6-82f6-32e01b1d2410 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:00.921919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ac933-71b1-4790-ae13-67a1a8f79a0b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:00.947310 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:00.965733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043ad5c-11e4-4004-8f72-80ce3b448339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:00.966128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebce07-aba4-4cc6-82f6-32e01b1d2410 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:00.966233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043ad5c-11e4-4004-8f72-80ce3b448339 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:00.966233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043ad5c-11e4-4004-8f72-80ce3b448339 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:00.966233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043ad5c-11e4-4004-8f72-80ce3b448339 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:00.966233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043ad5c-11e4-4004-8f72-80ce3b448339 None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:00.966233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043ad5c-11e4-4004-8f72-80ce3b448339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:00.966233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043ad5c-11e4-4004-8f72-80ce3b448339 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:00.979686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 149/279] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:00.981735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043ad5c-11e4-4004-8f72-80ce3b448339 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:00.995232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:01.016132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6bac1-a822-48ab-8149-81ba9fdf770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:01.017037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6bac1-a822-48ab-8149-81ba9fdf770e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.017441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6bac1-a822-48ab-8149-81ba9fdf770e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.017820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6bac1-a822-48ab-8149-81ba9fdf770e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.030840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6bac1-a822-48ab-8149-81ba9fdf770e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-fec6bac1-a822-48ab-8149-81ba9fdf770e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:01.038894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:01.039785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:01.040606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 131/280] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:01.047510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6bac1-a822-48ab-8149-81ba9fdf770e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:01.101381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043ad5c-11e4-4004-8f72-80ce3b448339 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:01.148206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac72424-628a-4f71-bf9e-642dc26797c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:01.149099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac72424-628a-4f71-bf9e-642dc26797c4 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.149384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac72424-628a-4f71-bf9e-642dc26797c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.149660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac72424-628a-4f71-bf9e-642dc26797c4 None admin] PATH_INFO: `/v3/users/a9094b51afec426e84307989bd1a1d15` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.168335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac72424-628a-4f71-bf9e-642dc26797c4 None admin] RBAC: Authorizing `identity:get_user(user_id=a9094b51afec426e84307989bd1a1d15)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:01.170016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac72424-628a-4f71-bf9e-642dc26797c4 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:01.174512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 150/281] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:00 2022] GET /identity/v3/users/a9094b51afec426e84307989bd1a1d15 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:01.178092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac72424-628a-4f71-bf9e-642dc26797c4 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:01.186323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6bac1-a822-48ab-8149-81ba9fdf770e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:01.228710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af066b-0d1b-4f47-b962-8c35291b5126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:01.229308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af066b-0d1b-4f47-b962-8c35291b5126 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.229604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af066b-0d1b-4f47-b962-8c35291b5126 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.229913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af066b-0d1b-4f47-b962-8c35291b5126 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.230766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af066b-0d1b-4f47-b962-8c35291b5126 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:01.231075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af066b-0d1b-4f47-b962-8c35291b5126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:01.232460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af066b-0d1b-4f47-b962-8c35291b5126 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:01.241172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 132/282] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:01.242588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa83b5-5bd5-4def-b555-047cc258850e None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.242874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa83b5-5bd5-4def-b555-047cc258850e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.243146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa83b5-5bd5-4def-b555-047cc258850e None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.244534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 133/283] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:01.249667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa83b5-5bd5-4def-b555-047cc258850e None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:01.267196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:01.274854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:01.300016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac72424-628a-4f71-bf9e-642dc26797c4 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:01.335868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd66a4-feca-4e1d-bd92-6c9c0f9daaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:01.336534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd66a4-feca-4e1d-bd92-6c9c0f9daaea None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.336822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd66a4-feca-4e1d-bd92-6c9c0f9daaea None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.337098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd66a4-feca-4e1d-bd92-6c9c0f9daaea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.337944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd66a4-feca-4e1d-bd92-6c9c0f9daaea None admin] RBAC: Adding query filter params (role.id=5dce8d92cffa40f79dfa1c03724c47e2, user.id=a9094b51afec426e84307989bd1a1d15, scope.system=all) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:01.338260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd66a4-feca-4e1d-bd92-6c9c0f9daaea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:01.339806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd66a4-feca-4e1d-bd92-6c9c0f9daaea None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:01.345487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 151/284] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Jul 25 05:40:01 2022] GET /identity/v3/role_assignments?user.id=a9094b51afec426e84307989bd1a1d15&scope.system=all&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:01.351668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406453d-7270-4072-955b-d7ee5609743c None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.351950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406453d-7270-4072-955b-d7ee5609743c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.352234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406453d-7270-4072-955b-d7ee5609743c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.356893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa83b5-5bd5-4def-b555-047cc258850e None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:01.383303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-6406453d-7270-4072-955b-d7ee5609743c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:01.384521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6406453d-7270-4072-955b-d7ee5609743c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:01.395645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0ebd0-fe83-4e17-a0ca-41a97f919c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:01.396580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0ebd0-fe83-4e17-a0ca-41a97f919c35 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.396914 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0ebd0-fe83-4e17-a0ca-41a97f919c35 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.397189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0ebd0-fe83-4e17-a0ca-41a97f919c35 None admin] PATH_INFO: `/v3/users/1b502b08b2544366beffe285fa1830f4` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.398450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0ebd0-fe83-4e17-a0ca-41a97f919c35 None admin] RBAC: Authorizing `identity:get_user(user_id=1b502b08b2544366beffe285fa1830f4)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:01.399948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0ebd0-fe83-4e17-a0ca-41a97f919c35 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:01.401997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 134/285] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:01 2022] GET /identity/v3/users/1b502b08b2544366beffe285fa1830f4 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:01.412051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0ebd0-fe83-4e17-a0ca-41a97f919c35 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:01.514988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:01.533492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0ebd0-fe83-4e17-a0ca-41a97f919c35 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:01.544936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 152/286] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:01.569674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406453d-7270-4072-955b-d7ee5609743c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:01.572263 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b9447-64fc-4aa6-91fe-a2ef8babfe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:01.573146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b9447-64fc-4aa6-91fe-a2ef8babfe80 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.573429 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b9447-64fc-4aa6-91fe-a2ef8babfe80 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.574681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b9447-64fc-4aa6-91fe-a2ef8babfe80 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.576091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b9447-64fc-4aa6-91fe-a2ef8babfe80 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:01.577616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b9447-64fc-4aa6-91fe-a2ef8babfe80 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:01.582600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 135/287] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:01 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:01.589302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b9447-64fc-4aa6-91fe-a2ef8babfe80 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:01.678826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:01.702744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b9447-64fc-4aa6-91fe-a2ef8babfe80 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:01.740496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c428f5-7853-4ce5-b81a-5cae83ca3e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:01.741495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c428f5-7853-4ce5-b81a-5cae83ca3e5b None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.742119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c428f5-7853-4ce5-b81a-5cae83ca3e5b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.742489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c428f5-7853-4ce5-b81a-5cae83ca3e5b None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0/users/1b502b08b2544366beffe285fa1830f4/roles/abb740756b544900822d2326908df69e` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.744548 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c428f5-7853-4ce5-b81a-5cae83ca3e5b None admin] RBAC: Authorizing `identity:create_grant(project_id=14afae0ea8a043bca3106a84957c35a0, user_id=1b502b08b2544366beffe285fa1830f4, role_id=abb740756b544900822d2326908df69e)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:01.747148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c428f5-7853-4ce5-b81a-5cae83ca3e5b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:01.765915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406453d-7270-4072-955b-d7ee5609743c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:01.802091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c63d10-f929-408a-822c-7194d97fde09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:01.803045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c63d10-f929-408a-822c-7194d97fde09 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.803686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c63d10-f929-408a-822c-7194d97fde09 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.804024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c63d10-f929-408a-822c-7194d97fde09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.814937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c63d10-f929-408a-822c-7194d97fde09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-25c63d10-f929-408a-822c-7194d97fde09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:01.823127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:01.824077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:01.824978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:01.829376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 153/288] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:01.831835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 136/289] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:01 2022] PUT /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0/users/1b502b08b2544366beffe285fa1830f4/roles/abb740756b544900822d2326908df69e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:01.832832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c63d10-f929-408a-822c-7194d97fde09 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:01.946634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c63d10-f929-408a-822c-7194d97fde09 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:01.971603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:01.989277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f2968-ea75-44f3-a644-ed32e42dd23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:01.989968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f2968-ea75-44f3-a644-ed32e42dd23a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.990303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f2968-ea75-44f3-a644-ed32e42dd23a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.990606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f2968-ea75-44f3-a644-ed32e42dd23a None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.991423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f2968-ea75-44f3-a644-ed32e42dd23a None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:01.991735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f2968-ea75-44f3-a644-ed32e42dd23a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:01.993167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f2968-ea75-44f3-a644-ed32e42dd23a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.008416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 154/290] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:02.013062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c428f5-7853-4ce5-b81a-5cae83ca3e5b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.027315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:02.122009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c428f5-7853-4ce5-b81a-5cae83ca3e5b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.164428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538280e-391a-43ab-8096-c2647a08b35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.165309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538280e-391a-43ab-8096-c2647a08b35b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.165590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538280e-391a-43ab-8096-c2647a08b35b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.165864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538280e-391a-43ab-8096-c2647a08b35b None admin] PATH_INFO: `/v3/users/b87a15e0c0054b9a8bcb1d285bfe9d1c` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.167093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538280e-391a-43ab-8096-c2647a08b35b None admin] RBAC: Authorizing `identity:get_user(user_id=b87a15e0c0054b9a8bcb1d285bfe9d1c)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.168575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538280e-391a-43ab-8096-c2647a08b35b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.170531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 137/291] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:02 2022] GET /identity/v3/users/b87a15e0c0054b9a8bcb1d285bfe9d1c => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:02.175866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f2968-ea75-44f3-a644-ed32e42dd23a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.327048 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f2968-ea75-44f3-a644-ed32e42dd23a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.374732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327e13f-648c-468b-b5f7-3bb1c1eb84f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.375406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e13f-648c-468b-b5f7-3bb1c1eb84f3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.375684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e13f-648c-468b-b5f7-3bb1c1eb84f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.375955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e13f-648c-468b-b5f7-3bb1c1eb84f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.446191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:02.473809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327e13f-648c-468b-b5f7-3bb1c1eb84f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.475379 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327e13f-648c-468b-b5f7-3bb1c1eb84f3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.605916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 155/292] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:02.611514 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538280e-391a-43ab-8096-c2647a08b35b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.727564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:02.758168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538280e-391a-43ab-8096-c2647a08b35b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.808040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead425e-d946-4120-afce-6ec5f4382761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.809446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead425e-d946-4120-afce-6ec5f4382761 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.809727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead425e-d946-4120-afce-6ec5f4382761 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.810001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead425e-d946-4120-afce-6ec5f4382761 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.811221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead425e-d946-4120-afce-6ec5f4382761 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.812670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead425e-d946-4120-afce-6ec5f4382761 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.814698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 138/293] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:02 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:02.820101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327e13f-648c-468b-b5f7-3bb1c1eb84f3 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.936105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:02.957968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327e13f-648c-468b-b5f7-3bb1c1eb84f3 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.994323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ffe8-681d-4219-b28c-408dcc672f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.995172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ffe8-681d-4219-b28c-408dcc672f78 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.995458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ffe8-681d-4219-b28c-408dcc672f78 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.995733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ffe8-681d-4219-b28c-408dcc672f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.997125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ffe8-681d-4219-b28c-408dcc672f78 None admin] RBAC: Adding query filter params (role.id=5dce8d92cffa40f79dfa1c03724c47e2, scope.project.id=14afae0ea8a043bca3106a84957c35a0, user.id=b87a15e0c0054b9a8bcb1d285bfe9d1c) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:02.997472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ffe8-681d-4219-b28c-408dcc672f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.998959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ffe8-681d-4219-b28c-408dcc672f78 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:03.005529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 156/294] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:02 2022] GET /identity/v3/role_assignments?user.id=b87a15e0c0054b9a8bcb1d285bfe9d1c&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:03.505684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50831fc2-cead-4a64-8cc0-eab899c6c3cf None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:03.505684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50831fc2-cead-4a64-8cc0-eab899c6c3cf None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:03.505684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50831fc2-cead-4a64-8cc0-eab899c6c3cf None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:03.506749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 139/295] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:03.512373 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138181d2-31e3-4234-b78c-ad344ce2f8d7 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:03.513149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138181d2-31e3-4234-b78c-ad344ce2f8d7 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:03.513880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138181d2-31e3-4234-b78c-ad344ce2f8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:03.532688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-138181d2-31e3-4234-b78c-ad344ce2f8d7 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:03.534207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-138181d2-31e3-4234-b78c-ad344ce2f8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:03.592302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666bd79-7cff-47a1-a905-20488be8bc5c None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:03.592625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666bd79-7cff-47a1-a905-20488be8bc5c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:03.592857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666bd79-7cff-47a1-a905-20488be8bc5c None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:03.594373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 140/296] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:03.600775 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212ce61-c065-4da1-a24f-26e6e3b19a6a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:03.601066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212ce61-c065-4da1-a24f-26e6e3b19a6a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:03.601349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212ce61-c065-4da1-a24f-26e6e3b19a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:03.614496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 157/297] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:03.628799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0212ce61-c065-4da1-a24f-26e6e3b19a6a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:03.630007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0212ce61-c065-4da1-a24f-26e6e3b19a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:03.637728 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138181d2-31e3-4234-b78c-ad344ce2f8d7 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:03.728256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 141/298] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:03.742867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:03.751039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212ce61-c065-4da1-a24f-26e6e3b19a6a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:03.773918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138181d2-31e3-4234-b78c-ad344ce2f8d7 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:03.819431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7276d-01fb-4adc-894e-d19b3e500ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:03.820597 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7276d-01fb-4adc-894e-d19b3e500ac9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:03.821109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7276d-01fb-4adc-894e-d19b3e500ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:03.821724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7276d-01fb-4adc-894e-d19b3e500ac9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:03.834791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:03.838325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7276d-01fb-4adc-894e-d19b3e500ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-50d7276d-01fb-4adc-894e-d19b3e500ac9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:03.849254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:03.850182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:03.851032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:03.858245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 158/299] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:03.860141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7276d-01fb-4adc-894e-d19b3e500ac9 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:03.869887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212ce61-c065-4da1-a24f-26e6e3b19a6a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:03.912661 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d66d67-e1b5-4b91-b06b-56dc75a97710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:03.913726 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d66d67-e1b5-4b91-b06b-56dc75a97710 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:03.914153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d66d67-e1b5-4b91-b06b-56dc75a97710 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:03.914600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d66d67-e1b5-4b91-b06b-56dc75a97710 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:03.915929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d66d67-e1b5-4b91-b06b-56dc75a97710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:03.917625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d66d67-e1b5-4b91-b06b-56dc75a97710 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:03.919580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 142/300] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:03.925212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d66d67-e1b5-4b91-b06b-56dc75a97710 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:03.983315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7276d-01fb-4adc-894e-d19b3e500ac9 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:04.037248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4177dd7-d085-4d09-b011-c32ff823e63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:04.037971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4177dd7-d085-4d09-b011-c32ff823e63e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.038258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4177dd7-d085-4d09-b011-c32ff823e63e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.038566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4177dd7-d085-4d09-b011-c32ff823e63e None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.039348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4177dd7-d085-4d09-b011-c32ff823e63e None admin] RBAC: Adding query filter params (name=reader) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:04.039793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4177dd7-d085-4d09-b011-c32ff823e63e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:04.041818 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4177dd7-d085-4d09-b011-c32ff823e63e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:04.044648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:04.051328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d66d67-e1b5-4b91-b06b-56dc75a97710 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:04.054664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 159/301] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:04.060163 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4177dd7-d085-4d09-b011-c32ff823e63e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:04.096612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15a96b-dd8c-4a3c-bbc0-194a355f1e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:04.097531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a96b-dd8c-4a3c-bbc0-194a355f1e07 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.098034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a96b-dd8c-4a3c-bbc0-194a355f1e07 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.098398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a96b-dd8c-4a3c-bbc0-194a355f1e07 None admin] PATH_INFO: `/v3/groups` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.099135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15a96b-dd8c-4a3c-bbc0-194a355f1e07 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:04.100648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15a96b-dd8c-4a3c-bbc0-194a355f1e07 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:04.164104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4177dd7-d085-4d09-b011-c32ff823e63e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:04.176439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 143/302] 10.222.0.17 () {60 vars in 1134 bytes} [Mon Jul 25 05:40:03 2022] POST /identity/v3/groups => generated 229 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:40:04.178194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1e338-e8b0-402e-b5f4-e21c81e1dda9 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.178194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1e338-e8b0-402e-b5f4-e21c81e1dda9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.180381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1e338-e8b0-402e-b5f4-e21c81e1dda9 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.180381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 144/303] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:04.186359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb5fa3-9db3-487a-a690-070c5600e81a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.186702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb5fa3-9db3-487a-a690-070c5600e81a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.186948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb5fa3-9db3-487a-a690-070c5600e81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.194729 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:04.205670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e6637-ee0c-44ab-9721-0ed076a488c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:04.206654 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e6637-ee0c-44ab-9721-0ed076a488c3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.206942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e6637-ee0c-44ab-9721-0ed076a488c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.207385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e6637-ee0c-44ab-9721-0ed076a488c3 None admin] PATH_INFO: `/v3/users/7d60132787e54403af56f43d966150bb` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.208292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-5feb5fa3-9db3-487a-a690-070c5600e81a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:04.209528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5feb5fa3-9db3-487a-a690-070c5600e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:04.223771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e6637-ee0c-44ab-9721-0ed076a488c3 None admin] RBAC: Authorizing `identity:get_user(user_id=7d60132787e54403af56f43d966150bb)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:04.225685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e6637-ee0c-44ab-9721-0ed076a488c3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:04.227756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 160/304] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:04 2022] GET /identity/v3/users/7d60132787e54403af56f43d966150bb => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:04.233790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e6637-ee0c-44ab-9721-0ed076a488c3 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:04.288576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 145/305] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:04.289889 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec12718-1954-49c6-960a-2b3fefdd22c2 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.290179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec12718-1954-49c6-960a-2b3fefdd22c2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.290456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec12718-1954-49c6-960a-2b3fefdd22c2 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.294605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 146/306] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:04.297563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666cadd-0607-4c1f-99d9-9a6946054558 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.297858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666cadd-0607-4c1f-99d9-9a6946054558 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.298118 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666cadd-0607-4c1f-99d9-9a6946054558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.315577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:04.322781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c666cadd-0607-4c1f-99d9-9a6946054558 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:04.324029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c666cadd-0607-4c1f-99d9-9a6946054558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:04.373459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e6637-ee0c-44ab-9721-0ed076a488c3 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:04.400929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 147/307] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:04.401593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666cadd-0607-4c1f-99d9-9a6946054558 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:04.411465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65e4a1-63b4-4a34-8223-2f23dfef6503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:04.412412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65e4a1-63b4-4a34-8223-2f23dfef6503 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.412890 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65e4a1-63b4-4a34-8223-2f23dfef6503 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.413396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65e4a1-63b4-4a34-8223-2f23dfef6503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.414301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65e4a1-63b4-4a34-8223-2f23dfef6503 None admin] RBAC: Adding query filter params (role.id=abb740756b544900822d2326908df69e, user.id=7d60132787e54403af56f43d966150bb, scope.system=all) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:04.414719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65e4a1-63b4-4a34-8223-2f23dfef6503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:04.416225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65e4a1-63b4-4a34-8223-2f23dfef6503 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:04.422464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 161/308] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Jul 25 05:40:04 2022] GET /identity/v3/role_assignments?user.id=7d60132787e54403af56f43d966150bb&scope.system=all&role.id=abb740756b544900822d2326908df69e => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:04.428623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65e4a1-63b4-4a34-8223-2f23dfef6503 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:04.491374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:04.540780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65e4a1-63b4-4a34-8223-2f23dfef6503 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:04.548017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666cadd-0607-4c1f-99d9-9a6946054558 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:04.570934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:04.583350 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6daaf-e71b-4ad5-909c-61638b00b27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:04.584175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6daaf-e71b-4ad5-909c-61638b00b27c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.584467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6daaf-e71b-4ad5-909c-61638b00b27c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.585436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6daaf-e71b-4ad5-909c-61638b00b27c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.601562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6daaf-e71b-4ad5-909c-61638b00b27c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:04.601913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd82245-ca05-4155-b392-7f41a62dd418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:04.603312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd82245-ca05-4155-b392-7f41a62dd418 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.603312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd82245-ca05-4155-b392-7f41a62dd418 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.603312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd82245-ca05-4155-b392-7f41a62dd418 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-37f6daaf-e71b-4ad5-909c-61638b00b27c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:04.611007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:04.612108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:04.613105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:04.617749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 162/309] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:04.620571 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd82245-ca05-4155-b392-7f41a62dd418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:04.623378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6daaf-e71b-4ad5-909c-61638b00b27c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-abd82245-ca05-4155-b392-7f41a62dd418 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:04.628904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:04.630020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:04.631039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:04.634611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 148/310] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:04.635960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd82245-ca05-4155-b392-7f41a62dd418 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:04.745549 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd82245-ca05-4155-b392-7f41a62dd418 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:04.758426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6daaf-e71b-4ad5-909c-61638b00b27c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:04.770182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:04.781511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d744f-25e1-41b4-89a5-a7e183477a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:04.790465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d744f-25e1-41b4-89a5-a7e183477a62 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.790465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d744f-25e1-41b4-89a5-a7e183477a62 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.790465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d744f-25e1-41b4-89a5-a7e183477a62 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.790465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d744f-25e1-41b4-89a5-a7e183477a62 None admin] RBAC: Adding query filter params (name=member) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:04.790465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d744f-25e1-41b4-89a5-a7e183477a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:04.790465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d744f-25e1-41b4-89a5-a7e183477a62 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:04.798261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bf6a3-a1bd-4cd6-9007-834d34d349c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:04.798860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 149/311] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:04.799553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bf6a3-a1bd-4cd6-9007-834d34d349c2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.799782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d744f-25e1-41b4-89a5-a7e183477a62 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:04.800252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bf6a3-a1bd-4cd6-9007-834d34d349c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.800674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bf6a3-a1bd-4cd6-9007-834d34d349c2 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.801570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bf6a3-a1bd-4cd6-9007-834d34d349c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:04.802022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bf6a3-a1bd-4cd6-9007-834d34d349c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:04.803609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bf6a3-a1bd-4cd6-9007-834d34d349c2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:04.814736 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 163/312] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:04.819917 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bf6a3-a1bd-4cd6-9007-834d34d349c2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:04.878420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:04.924555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d744f-25e1-41b4-89a5-a7e183477a62 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:04.961289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab51f8-26d8-4f9a-ad4c-58848c6a39b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:04.962174 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab51f8-26d8-4f9a-ad4c-58848c6a39b2 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:04.962511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab51f8-26d8-4f9a-ad4c-58848c6a39b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:04.962786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab51f8-26d8-4f9a-ad4c-58848c6a39b2 None admin] PATH_INFO: `/v3/users/a9094b51afec426e84307989bd1a1d15` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:04.964302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab51f8-26d8-4f9a-ad4c-58848c6a39b2 None admin] RBAC: Authorizing `identity:get_user(user_id=a9094b51afec426e84307989bd1a1d15)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:04.965823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab51f8-26d8-4f9a-ad4c-58848c6a39b2 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:04.967495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 150/313] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:04 2022] GET /identity/v3/users/a9094b51afec426e84307989bd1a1d15 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:04.973047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab51f8-26d8-4f9a-ad4c-58848c6a39b2 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:04.980740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bf6a3-a1bd-4cd6-9007-834d34d349c2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:05.024923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab88c7-c715-489b-8162-7b8a4126abff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:05.026855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab88c7-c715-489b-8162-7b8a4126abff None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:05.027393 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab88c7-c715-489b-8162-7b8a4126abff None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:05.028078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab88c7-c715-489b-8162-7b8a4126abff None admin] PATH_INFO: `/v3/users/1b502b08b2544366beffe285fa1830f4` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:05.030104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab88c7-c715-489b-8162-7b8a4126abff None admin] RBAC: Authorizing `identity:get_user(user_id=1b502b08b2544366beffe285fa1830f4)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:05.032516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:05.032821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab88c7-c715-489b-8162-7b8a4126abff None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:05.040836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 164/314] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:04 2022] GET /identity/v3/users/1b502b08b2544366beffe285fa1830f4 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:05.041554 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab88c7-c715-489b-8162-7b8a4126abff None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:05.086063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab51f8-26d8-4f9a-ad4c-58848c6a39b2 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:05.123523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba3644-5f6b-42f9-88c6-6dd98e25ed07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:05.124508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba3644-5f6b-42f9-88c6-6dd98e25ed07 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:05.124851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba3644-5f6b-42f9-88c6-6dd98e25ed07 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:05.125192 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba3644-5f6b-42f9-88c6-6dd98e25ed07 None admin] PATH_INFO: `/v3/system/users/a9094b51afec426e84307989bd1a1d15/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:05.126848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba3644-5f6b-42f9-88c6-6dd98e25ed07 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a9094b51afec426e84307989bd1a1d15, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:05.128391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba3644-5f6b-42f9-88c6-6dd98e25ed07 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:05.163406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab88c7-c715-489b-8162-7b8a4126abff None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:05.174988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:05.199023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88f77c-6d6c-406e-aa0d-e527dd0827bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:05.199710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88f77c-6d6c-406e-aa0d-e527dd0827bd None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:05.199999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88f77c-6d6c-406e-aa0d-e527dd0827bd None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:05.200280 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88f77c-6d6c-406e-aa0d-e527dd0827bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:05.243136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 151/315] 10.222.0.17 () {58 vars in 1334 bytes} [Mon Jul 25 05:40:04 2022] PUT /identity/v3/system/users/a9094b51afec426e84307989bd1a1d15/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:40:05.280512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88f77c-6d6c-406e-aa0d-e527dd0827bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:05.282858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88f77c-6d6c-406e-aa0d-e527dd0827bd None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:05.415077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 165/316] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:40:05.421786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba3644-5f6b-42f9-88c6-6dd98e25ed07 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:05.530096 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba3644-5f6b-42f9-88c6-6dd98e25ed07 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:05.547992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:05.565553 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04e2e7-cc67-4089-ba84-dd5adaea62d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:05.566440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04e2e7-cc67-4089-ba84-dd5adaea62d4 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:05.566804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04e2e7-cc67-4089-ba84-dd5adaea62d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:05.567078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04e2e7-cc67-4089-ba84-dd5adaea62d4 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:05.568314 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04e2e7-cc67-4089-ba84-dd5adaea62d4 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:05.569802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04e2e7-cc67-4089-ba84-dd5adaea62d4 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:05.571972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 152/317] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:05 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:05.577888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88f77c-6d6c-406e-aa0d-e527dd0827bd None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:05.592738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:05.689811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88f77c-6d6c-406e-aa0d-e527dd0827bd None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:05.724829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7f426-1f60-47bb-8844-729e5cbafd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:05.725770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7f426-1f60-47bb-8844-729e5cbafd81 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:05.726255 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7f426-1f60-47bb-8844-729e5cbafd81 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:05.726749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7f426-1f60-47bb-8844-729e5cbafd81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:05.727756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7f426-1f60-47bb-8844-729e5cbafd81 None admin] RBAC: Adding query filter params (role.id=abb740756b544900822d2326908df69e, scope.project.id=14afae0ea8a043bca3106a84957c35a0, user.id=1b502b08b2544366beffe285fa1830f4) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:05.728238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7f426-1f60-47bb-8844-729e5cbafd81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:05.729849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7f426-1f60-47bb-8844-729e5cbafd81 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:05.737108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 166/318] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:05 2022] GET /identity/v3/role_assignments?user.id=1b502b08b2544366beffe285fa1830f4&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=abb740756b544900822d2326908df69e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:06.308371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214953c-4ef6-41d9-b1cb-5025e8063902 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:06.308744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214953c-4ef6-41d9-b1cb-5025e8063902 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:06.308964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214953c-4ef6-41d9-b1cb-5025e8063902 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:06.310523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 153/319] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:06.315621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fe319-0bee-4d4a-b022-b43a3e7c8dd5 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:06.315921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fe319-0bee-4d4a-b022-b43a3e7c8dd5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:06.316205 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fe319-0bee-4d4a-b022-b43a3e7c8dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:06.333320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-254fe319-0bee-4d4a-b022-b43a3e7c8dd5 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:06.334546 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-254fe319-0bee-4d4a-b022-b43a3e7c8dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:06.403957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 167/320] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:06.426938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214953c-4ef6-41d9-b1cb-5025e8063902 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:06.511629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbd0e9-98bf-471f-9b1a-125069cb9fec None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:06.511990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbd0e9-98bf-471f-9b1a-125069cb9fec None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:06.512270 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbd0e9-98bf-471f-9b1a-125069cb9fec None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:06.514037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 168/321] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:06.519843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3e32a-6314-4b7d-ab69-fd7f8b351d4e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:06.519843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3e32a-6314-4b7d-ab69-fd7f8b351d4e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:06.519843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3e32a-6314-4b7d-ab69-fd7f8b351d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:06.526192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:06.537243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d1e3e32a-6314-4b7d-ab69-fd7f8b351d4e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:06.541978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d1e3e32a-6314-4b7d-ab69-fd7f8b351d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:06.542080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214953c-4ef6-41d9-b1cb-5025e8063902 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:06.574573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d9ebd-1fd3-48ee-ab51-5747c1fb8a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:06.575294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d9ebd-1fd3-48ee-ab51-5747c1fb8a06 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:06.575579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d9ebd-1fd3-48ee-ab51-5747c1fb8a06 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:06.575854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d9ebd-1fd3-48ee-ab51-5747c1fb8a06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:06.577009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d9ebd-1fd3-48ee-ab51-5747c1fb8a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:06.578444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d9ebd-1fd3-48ee-ab51-5747c1fb8a06 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:06.580077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 154/322] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:06.585374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d9ebd-1fd3-48ee-ab51-5747c1fb8a06 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:06.612465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 169/323] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:06.634277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3e32a-6314-4b7d-ab69-fd7f8b351d4e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:06.644007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:06.690791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d9ebd-1fd3-48ee-ab51-5747c1fb8a06 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:06.724400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d413aeb-398e-42bc-86f0-2d0b6ca9dc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:06.725068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d413aeb-398e-42bc-86f0-2d0b6ca9dc6f None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:06.725354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d413aeb-398e-42bc-86f0-2d0b6ca9dc6f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:06.725635 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d413aeb-398e-42bc-86f0-2d0b6ca9dc6f None admin] PATH_INFO: `/v3/groups` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:06.726326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d413aeb-398e-42bc-86f0-2d0b6ca9dc6f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:06.727795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d413aeb-398e-42bc-86f0-2d0b6ca9dc6f None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:06.741527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3e32a-6314-4b7d-ab69-fd7f8b351d4e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:06.774676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39fcbf-9d74-4edb-8902-8b655fb95849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:06.775449 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39fcbf-9d74-4edb-8902-8b655fb95849 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:06.775733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39fcbf-9d74-4edb-8902-8b655fb95849 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:06.776002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39fcbf-9d74-4edb-8902-8b655fb95849 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:06.778384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:06.786583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39fcbf-9d74-4edb-8902-8b655fb95849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-be39fcbf-9d74-4edb-8902-8b655fb95849 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:06.794296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:06.795259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:06.796068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:06.798626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 170/324] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:06.800957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39fcbf-9d74-4edb-8902-8b655fb95849 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:06.810353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 155/325] 10.222.0.17 () {60 vars in 1134 bytes} [Mon Jul 25 05:40:06 2022] POST /identity/v3/groups => generated 226 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:40:06.922245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39fcbf-9d74-4edb-8902-8b655fb95849 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:06.960046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f0300-d488-41d1-97d4-040269839719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:06.960046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f0300-d488-41d1-97d4-040269839719 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:06.960333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f0300-d488-41d1-97d4-040269839719 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:06.960562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f0300-d488-41d1-97d4-040269839719 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:06.962175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f0300-d488-41d1-97d4-040269839719 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:06.962175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f0300-d488-41d1-97d4-040269839719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:06.963690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f0300-d488-41d1-97d4-040269839719 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:06.972775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 171/326] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:06.978425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d413aeb-398e-42bc-86f0-2d0b6ca9dc6f None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:07.092872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d413aeb-398e-42bc-86f0-2d0b6ca9dc6f None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:07.107485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:07.128857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6a7d7-a1f6-4615-a055-cfb5c403b984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:07.137317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6a7d7-a1f6-4615-a055-cfb5c403b984 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:07.137317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6a7d7-a1f6-4615-a055-cfb5c403b984 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:07.137317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6a7d7-a1f6-4615-a055-cfb5c403b984 None admin] PATH_INFO: `/v3/users/7d60132787e54403af56f43d966150bb` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:07.137317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6a7d7-a1f6-4615-a055-cfb5c403b984 None admin] RBAC: Authorizing `identity:get_user(user_id=7d60132787e54403af56f43d966150bb)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:07.137317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6a7d7-a1f6-4615-a055-cfb5c403b984 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:07.137317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 156/327] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:06 2022] GET /identity/v3/users/7d60132787e54403af56f43d966150bb => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:07.141563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f0300-d488-41d1-97d4-040269839719 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:07.212859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:07.270505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f0300-d488-41d1-97d4-040269839719 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:07.302626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150975a-ae37-4f54-a3a0-e65afbd686a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:07.303503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150975a-ae37-4f54-a3a0-e65afbd686a4 None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:07.303778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150975a-ae37-4f54-a3a0-e65afbd686a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:07.304052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150975a-ae37-4f54-a3a0-e65afbd686a4 None admin] PATH_INFO: `/v3/system/users/7d60132787e54403af56f43d966150bb/roles/abb740756b544900822d2326908df69e` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:07.306235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150975a-ae37-4f54-a3a0-e65afbd686a4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7d60132787e54403af56f43d966150bb, role_id=abb740756b544900822d2326908df69e)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:07.307910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150975a-ae37-4f54-a3a0-e65afbd686a4 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:07.380438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 172/328] 10.222.0.17 () {58 vars in 1334 bytes} [Mon Jul 25 05:40:07 2022] PUT /identity/v3/system/users/7d60132787e54403af56f43d966150bb/roles/abb740756b544900822d2326908df69e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:07.413947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5953e-244b-4525-b480-56467ce9e7f4 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:07.414275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5953e-244b-4525-b480-56467ce9e7f4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:07.414517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5953e-244b-4525-b480-56467ce9e7f4 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:07.416203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 157/329] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:07.423338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626137-fce5-4d3f-8503-f3a83cb3fdbc None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:07.423638 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626137-fce5-4d3f-8503-f3a83cb3fdbc None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:07.424521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0626137-fce5-4d3f-8503-f3a83cb3fdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:07.447647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-b0626137-fce5-4d3f-8503-f3a83cb3fdbc None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:07.448910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b0626137-fce5-4d3f-8503-f3a83cb3fdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:07.595561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 173/330] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:07.623324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5953e-244b-4525-b480-56467ce9e7f4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:07.742401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5953e-244b-4525-b480-56467ce9e7f4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:07.779057 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5120a-38a4-4e72-b476-7d5a3ccaf140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:07.780117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5120a-38a4-4e72-b476-7d5a3ccaf140 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:07.780568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5120a-38a4-4e72-b476-7d5a3ccaf140 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:07.780996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5120a-38a4-4e72-b476-7d5a3ccaf140 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:07.792051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5120a-38a4-4e72-b476-7d5a3ccaf140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-a1c5120a-38a4-4e72-b476-7d5a3ccaf140 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:07.800585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:07.801698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:07.802725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:07.805040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 158/331] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:07.811044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0626137-fce5-4d3f-8503-f3a83cb3fdbc None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:07.865199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:07.930960 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0626137-fce5-4d3f-8503-f3a83cb3fdbc None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:07.974580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06491e26-8634-4cb9-9506-15c0c3d58f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:07.975312 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06491e26-8634-4cb9-9506-15c0c3d58f76 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:07.975599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06491e26-8634-4cb9-9506-15c0c3d58f76 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:07.975878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06491e26-8634-4cb9-9506-15c0c3d58f76 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:07.976642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06491e26-8634-4cb9-9506-15c0c3d58f76 None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:07.976952 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06491e26-8634-4cb9-9506-15c0c3d58f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:07.978360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06491e26-8634-4cb9-9506-15c0c3d58f76 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:07.988237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 174/332] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:07.993798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5120a-38a4-4e72-b476-7d5a3ccaf140 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:08.040134 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:08.114427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5120a-38a4-4e72-b476-7d5a3ccaf140 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:08.154375 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb929683-6f3b-4aa2-8f30-e5fcb75dc121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:08.155221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb929683-6f3b-4aa2-8f30-e5fcb75dc121 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:08.155500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb929683-6f3b-4aa2-8f30-e5fcb75dc121 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:08.155810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb929683-6f3b-4aa2-8f30-e5fcb75dc121 None admin] PATH_INFO: `/v3/users/a9094b51afec426e84307989bd1a1d15` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:08.157546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb929683-6f3b-4aa2-8f30-e5fcb75dc121 None admin] RBAC: Authorizing `identity:get_user(user_id=a9094b51afec426e84307989bd1a1d15)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:08.159272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb929683-6f3b-4aa2-8f30-e5fcb75dc121 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:08.161200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 159/333] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:07 2022] GET /identity/v3/users/a9094b51afec426e84307989bd1a1d15 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:08.166702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06491e26-8634-4cb9-9506-15c0c3d58f76 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:08.291729 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06491e26-8634-4cb9-9506-15c0c3d58f76 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:08.305271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:08.337996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5966f52-1447-45c0-b3ed-180d9b20351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:08.338701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5966f52-1447-45c0-b3ed-180d9b20351d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:08.338986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5966f52-1447-45c0-b3ed-180d9b20351d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:08.339311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5966f52-1447-45c0-b3ed-180d9b20351d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:08.340723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5966f52-1447-45c0-b3ed-180d9b20351d None admin] RBAC: Adding query filter params (role.id=5dce8d92cffa40f79dfa1c03724c47e2, user.id=a9094b51afec426e84307989bd1a1d15, scope.system=all) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:08.341040 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5966f52-1447-45c0-b3ed-180d9b20351d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:08.342529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5966f52-1447-45c0-b3ed-180d9b20351d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:08.358886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 175/334] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Jul 25 05:40:08 2022] GET /identity/v3/role_assignments?user.id=a9094b51afec426e84307989bd1a1d15&scope.system=all&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:09.537628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256451f-c85e-4dc9-9930-2fc8dcbd8201 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:09.538142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256451f-c85e-4dc9-9930-2fc8dcbd8201 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:09.538251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256451f-c85e-4dc9-9930-2fc8dcbd8201 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:09.539857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 160/335] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:09.545385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dcbe4-dd3d-4a77-a77a-591e0ef1b762 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:09.545814 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dcbe4-dd3d-4a77-a77a-591e0ef1b762 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:09.546009 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dcbe4-dd3d-4a77-a77a-591e0ef1b762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:09.567234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-5f4dcbe4-dd3d-4a77-a77a-591e0ef1b762 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:09.568495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5f4dcbe4-dd3d-4a77-a77a-591e0ef1b762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:09.639735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23979977-e981-4614-b23b-15b21591379a None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:09.640053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23979977-e981-4614-b23b-15b21591379a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:09.640294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23979977-e981-4614-b23b-15b21591379a None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:09.642067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 161/336] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:09.647743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac6f69-3b78-4f39-9112-a315458597c9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:09.648050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac6f69-3b78-4f39-9112-a315458597c9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:09.648301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac6f69-3b78-4f39-9112-a315458597c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:09.661401 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 176/337] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:09.686411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-caac6f69-3b78-4f39-9112-a315458597c9 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:09.688017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-caac6f69-3b78-4f39-9112-a315458597c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:09.716044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dcbe4-dd3d-4a77-a77a-591e0ef1b762 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:09.834173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 162/338] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:09.899126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac6f69-3b78-4f39-9112-a315458597c9 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:09.900227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dcbe4-dd3d-4a77-a77a-591e0ef1b762 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:09.901972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:09.946784 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c61045-b0cb-4804-83bb-d08007a6ae82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:09.948313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c61045-b0cb-4804-83bb-d08007a6ae82 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:09.948847 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c61045-b0cb-4804-83bb-d08007a6ae82 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:09.949516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c61045-b0cb-4804-83bb-d08007a6ae82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:09.963850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c61045-b0cb-4804-83bb-d08007a6ae82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-38c61045-b0cb-4804-83bb-d08007a6ae82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:09.973711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:09.974647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:09.975840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 177/339] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:09.987290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c61045-b0cb-4804-83bb-d08007a6ae82 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:09.998837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:10.106034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac6f69-3b78-4f39-9112-a315458597c9 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:10.161094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520335c-5a1c-4f2a-a405-d64dd96a8ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:10.161908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520335c-5a1c-4f2a-a405-d64dd96a8ff2 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.162191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520335c-5a1c-4f2a-a405-d64dd96a8ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.162512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520335c-5a1c-4f2a-a405-d64dd96a8ff2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.171837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c61045-b0cb-4804-83bb-d08007a6ae82 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:10.176072 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520335c-5a1c-4f2a-a405-d64dd96a8ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-c520335c-5a1c-4f2a-a405-d64dd96a8ff2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:10.185267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:10.186211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:10.187068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:10.188101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 163/340] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:10.190256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4420ce2-9fe7-4d02-8bfe-ace6974e3749 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.190256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4420ce2-9fe7-4d02-8bfe-ace6974e3749 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.190256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4420ce2-9fe7-4d02-8bfe-ace6974e3749 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.191671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 164/341] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:10.196264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4420ce2-9fe7-4d02-8bfe-ace6974e3749 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:10.216167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf00e8-ce2e-4051-8d3d-2b6ad6090411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:10.217232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf00e8-ce2e-4051-8d3d-2b6ad6090411 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.217692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf00e8-ce2e-4051-8d3d-2b6ad6090411 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.218278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf00e8-ce2e-4051-8d3d-2b6ad6090411 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.219249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf00e8-ce2e-4051-8d3d-2b6ad6090411 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:10.219734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf00e8-ce2e-4051-8d3d-2b6ad6090411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:10.221441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf00e8-ce2e-4051-8d3d-2b6ad6090411 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:10.234183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 178/342] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:10.235786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13185e15-b18b-44b6-b490-8f28c4abd8a6 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.236273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13185e15-b18b-44b6-b490-8f28c4abd8a6 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.236724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13185e15-b18b-44b6-b490-8f28c4abd8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.260216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-13185e15-b18b-44b6-b490-8f28c4abd8a6 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:10.261949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-13185e15-b18b-44b6-b490-8f28c4abd8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:10.284588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:10.334045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:10.338443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4420ce2-9fe7-4d02-8bfe-ace6974e3749 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:10.342980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 179/343] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:10.343904 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13185e15-b18b-44b6-b490-8f28c4abd8a6 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:10.399225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae684e-ab8c-424e-a4af-f47b04281bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:10.399821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae684e-ab8c-424e-a4af-f47b04281bef None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.400105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae684e-ab8c-424e-a4af-f47b04281bef None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.400382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae684e-ab8c-424e-a4af-f47b04281bef None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.401238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae684e-ab8c-424e-a4af-f47b04281bef None admin] RBAC: Adding query filter params (name=member) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:10.401552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae684e-ab8c-424e-a4af-f47b04281bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:10.403621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae684e-ab8c-424e-a4af-f47b04281bef None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:10.415583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 165/344] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:10.416519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae684e-ab8c-424e-a4af-f47b04281bef None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:10.456291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13185e15-b18b-44b6-b490-8f28c4abd8a6 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:10.493500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28dfbb5-1db1-417a-acc5-708f886e56b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:10.494156 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28dfbb5-1db1-417a-acc5-708f886e56b7 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.494438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28dfbb5-1db1-417a-acc5-708f886e56b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.494806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28dfbb5-1db1-417a-acc5-708f886e56b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.600976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:10.621315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae684e-ab8c-424e-a4af-f47b04281bef None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:10.690865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74e2ed-a7df-494a-b190-1dc451c13989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:10.691601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74e2ed-a7df-494a-b190-1dc451c13989 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.691882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74e2ed-a7df-494a-b190-1dc451c13989 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.692157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74e2ed-a7df-494a-b190-1dc451c13989 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.693613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28dfbb5-1db1-417a-acc5-708f886e56b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:10.695347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28dfbb5-1db1-417a-acc5-708f886e56b7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:10.721226 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74e2ed-a7df-494a-b190-1dc451c13989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-4c74e2ed-a7df-494a-b190-1dc451c13989 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:10.739253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:10.740232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:10.741166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:10.743337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 166/345] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:10.746822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74e2ed-a7df-494a-b190-1dc451c13989 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:10.788955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:10.864561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 180/346] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:10.869789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28dfbb5-1db1-417a-acc5-708f886e56b7 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:10.873708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74e2ed-a7df-494a-b190-1dc451c13989 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:10.922331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8535865-f369-42cd-80b4-01d47785e238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:10.923067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8535865-f369-42cd-80b4-01d47785e238 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.923350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8535865-f369-42cd-80b4-01d47785e238 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.923621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8535865-f369-42cd-80b4-01d47785e238 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.983138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:11.030127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8535865-f369-42cd-80b4-01d47785e238 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:11.031650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8535865-f369-42cd-80b4-01d47785e238 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:11.036900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28dfbb5-1db1-417a-acc5-708f886e56b7 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:11.059514 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:11.079263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7ed0-c160-4f31-b816-f4542beb93cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:11.080305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7ed0-c160-4f31-b816-f4542beb93cf None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.080853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7ed0-c160-4f31-b816-f4542beb93cf None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.081339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7ed0-c160-4f31-b816-f4542beb93cf None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.082768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7ed0-c160-4f31-b816-f4542beb93cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:11.083305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7ed0-c160-4f31-b816-f4542beb93cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:11.085016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7ed0-c160-4f31-b816-f4542beb93cf None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:11.098210 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 181/347] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:11.104649 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7ed0-c160-4f31-b816-f4542beb93cf None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:11.155987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 167/348] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:11.158292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cfaff-fc42-4af3-abda-1d3ab49740d9 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.158849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cfaff-fc42-4af3-abda-1d3ab49740d9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.159858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cfaff-fc42-4af3-abda-1d3ab49740d9 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.162080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 168/349] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:11.163075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cfaff-fc42-4af3-abda-1d3ab49740d9 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:11.232302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7ed0-c160-4f31-b816-f4542beb93cf None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:11.251731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:11.274891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148fed8-dc40-4ab9-b7bf-aba62712dec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:11.276196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148fed8-dc40-4ab9-b7bf-aba62712dec2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.276677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148fed8-dc40-4ab9-b7bf-aba62712dec2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.277174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148fed8-dc40-4ab9-b7bf-aba62712dec2 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.281190 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148fed8-dc40-4ab9-b7bf-aba62712dec2 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:11.283031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148fed8-dc40-4ab9-b7bf-aba62712dec2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:11.285389 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 182/350] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:11 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:11.286549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148fed8-dc40-4ab9-b7bf-aba62712dec2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:11.309506 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cfaff-fc42-4af3-abda-1d3ab49740d9 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:11.334717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:11.347906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d698d-6e61-49ad-bf39-66330b6ed555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:11.348767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d698d-6e61-49ad-bf39-66330b6ed555 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.349047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d698d-6e61-49ad-bf39-66330b6ed555 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.349317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d698d-6e61-49ad-bf39-66330b6ed555 None admin] PATH_INFO: `/v3/users/7d60132787e54403af56f43d966150bb` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.350666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d698d-6e61-49ad-bf39-66330b6ed555 None admin] RBAC: Authorizing `identity:get_user(user_id=7d60132787e54403af56f43d966150bb)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:11.352285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d698d-6e61-49ad-bf39-66330b6ed555 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:11.354238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 169/351] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:11 2022] GET /identity/v3/users/7d60132787e54403af56f43d966150bb => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:11.355621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2c9be-7a96-4827-8621-e41e09f297af None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.355843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2c9be-7a96-4827-8621-e41e09f297af None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.356114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2c9be-7a96-4827-8621-e41e09f297af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.374725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-5db2c9be-7a96-4827-8621-e41e09f297af None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:11.376048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5db2c9be-7a96-4827-8621-e41e09f297af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:11.405171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148fed8-dc40-4ab9-b7bf-aba62712dec2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:11.452744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 170/352] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:11.453312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2c9be-7a96-4827-8621-e41e09f297af None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:11.456532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cccd31-66c2-4311-886c-f2063ca43fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:11.456532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cccd31-66c2-4311-886c-f2063ca43fa2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.456532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cccd31-66c2-4311-886c-f2063ca43fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.457058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cccd31-66c2-4311-886c-f2063ca43fa2 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.459278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cccd31-66c2-4311-886c-f2063ca43fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:11.460738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cccd31-66c2-4311-886c-f2063ca43fa2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:11.467192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 183/353] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:11 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:11.468514 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea9384-1753-4568-bba3-a0d02619a8bf None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.468834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea9384-1753-4568-bba3-a0d02619a8bf None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.469112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea9384-1753-4568-bba3-a0d02619a8bf None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.471217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 184/354] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:11.475139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea9384-1753-4568-bba3-a0d02619a8bf None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:11.540639 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:11.586873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2c9be-7a96-4827-8621-e41e09f297af None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:11.610376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:11.621572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea9384-1753-4568-bba3-a0d02619a8bf None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:11.637537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a1c02-c977-4657-848d-fe372a359f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:11.638894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a1c02-c977-4657-848d-fe372a359f4f None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.639091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a1c02-c977-4657-848d-fe372a359f4f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.639364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a1c02-c977-4657-848d-fe372a359f4f None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.648196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a1c02-c977-4657-848d-fe372a359f4f None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:11.649905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a1c02-c977-4657-848d-fe372a359f4f None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:11.652624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 171/355] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:11 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:11.653648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a1c02-c977-4657-848d-fe372a359f4f None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:11.667583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1f4ad-a86f-45f1-b1de-cd9e3aca76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:11.667583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1f4ad-a86f-45f1-b1de-cd9e3aca76b9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.668015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1f4ad-a86f-45f1-b1de-cd9e3aca76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.668264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1f4ad-a86f-45f1-b1de-cd9e3aca76b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.669146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1f4ad-a86f-45f1-b1de-cd9e3aca76b9 None admin] RBAC: Adding query filter params (role.id=abb740756b544900822d2326908df69e, user.id=7d60132787e54403af56f43d966150bb, scope.system=all) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:11.669463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1f4ad-a86f-45f1-b1de-cd9e3aca76b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:11.671498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1f4ad-a86f-45f1-b1de-cd9e3aca76b9 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:11.694905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 185/356] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Jul 25 05:40:11 2022] GET /identity/v3/role_assignments?user.id=7d60132787e54403af56f43d966150bb&scope.system=all&role.id=abb740756b544900822d2326908df69e => generated 510 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:11.697772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81577a7-8a97-458f-b828-635829b15c46 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.698582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81577a7-8a97-458f-b828-635829b15c46 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.699665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81577a7-8a97-458f-b828-635829b15c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.718060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e81577a7-8a97-458f-b828-635829b15c46 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:11.719833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e81577a7-8a97-458f-b828-635829b15c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:11.775762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a1c02-c977-4657-848d-fe372a359f4f None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:11.799376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 186/357] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:11.800619 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81577a7-8a97-458f-b828-635829b15c46 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:11.819297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fb93d-49be-4b12-b5e9-460c4c2b3ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:11.820464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fb93d-49be-4b12-b5e9-460c4c2b3ab8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.820747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fb93d-49be-4b12-b5e9-460c4c2b3ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.821021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fb93d-49be-4b12-b5e9-460c4c2b3ab8 None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.822761 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fb93d-49be-4b12-b5e9-460c4c2b3ab8 None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:11.824816 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fb93d-49be-4b12-b5e9-460c4c2b3ab8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:11.835660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 172/358] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:11 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:11.836566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fb93d-49be-4b12-b5e9-460c4c2b3ab8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:11.861975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:11.925765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:11.939816 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81577a7-8a97-458f-b828-635829b15c46 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:11.981093 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825b201-7252-47bd-b4f7-454f87005b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:11.982384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825b201-7252-47bd-b4f7-454f87005b2c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:11.982702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825b201-7252-47bd-b4f7-454f87005b2c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:11.983252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825b201-7252-47bd-b4f7-454f87005b2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:11.989893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fb93d-49be-4b12-b5e9-460c4c2b3ab8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:11.995413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825b201-7252-47bd-b4f7-454f87005b2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-6825b201-7252-47bd-b4f7-454f87005b2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:12.005153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:12.006092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:12.006940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 187/359] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:12.008226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825b201-7252-47bd-b4f7-454f87005b2c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.036857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72084ef3-0434-4b34-8ff3-3a40f06c09df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.037519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72084ef3-0434-4b34-8ff3-3a40f06c09df None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.037799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72084ef3-0434-4b34-8ff3-3a40f06c09df None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.038077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72084ef3-0434-4b34-8ff3-3a40f06c09df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.038942 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72084ef3-0434-4b34-8ff3-3a40f06c09df None admin] RBAC: Adding query filter params (group.id=e902269caaf94142a68bebe80b795809, role.id=73a019c7fb8340a58c7dfe4550341949, scope.project.id=56fabc309a30484ebda9a05789fa5271) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:12.039255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72084ef3-0434-4b34-8ff3-3a40f06c09df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:12.040693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72084ef3-0434-4b34-8ff3-3a40f06c09df None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:12.047984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 173/360] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 05:40:11 2022] GET /identity/v3/role_assignments?group.id=e902269caaf94142a68bebe80b795809&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=73a019c7fb8340a58c7dfe4550341949 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:12.048731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72084ef3-0434-4b34-8ff3-3a40f06c09df None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.137448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:12.143239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825b201-7252-47bd-b4f7-454f87005b2c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.192262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e6d1a-1d3a-4086-ba33-d8be39470a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.193078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6d1a-1d3a-4086-ba33-d8be39470a58 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.193433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6d1a-1d3a-4086-ba33-d8be39470a58 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.193780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6d1a-1d3a-4086-ba33-d8be39470a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.194765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e6d1a-1d3a-4086-ba33-d8be39470a58 None admin] RBAC: Adding query filter params (group.id=e902269caaf94142a68bebe80b795809, role.id=5dce8d92cffa40f79dfa1c03724c47e2, scope.project.id=56fabc309a30484ebda9a05789fa5271) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:12.195084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e6d1a-1d3a-4086-ba33-d8be39470a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:12.196588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e6d1a-1d3a-4086-ba33-d8be39470a58 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:12.203255 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 188/361] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/role_assignments?group.id=e902269caaf94142a68bebe80b795809&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:12.204587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb5f16-b344-4f5a-9111-52fdbdcd5e3c None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.204876 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb5f16-b344-4f5a-9111-52fdbdcd5e3c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.205153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb5f16-b344-4f5a-9111-52fdbdcd5e3c None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.206610 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 189/362] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:12.207324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb5f16-b344-4f5a-9111-52fdbdcd5e3c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.210332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72084ef3-0434-4b34-8ff3-3a40f06c09df None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.235447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:12.259048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386679e6-42bc-4e98-b980-7a7153ec6fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.259907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386679e6-42bc-4e98-b980-7a7153ec6fbd None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.260285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386679e6-42bc-4e98-b980-7a7153ec6fbd None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.260572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386679e6-42bc-4e98-b980-7a7153ec6fbd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.274979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386679e6-42bc-4e98-b980-7a7153ec6fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-386679e6-42bc-4e98-b980-7a7153ec6fbd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:12.284851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:12.285839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:12.286707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:12.290577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 174/363] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:12.292049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d89f7-d2a0-48bc-b9ea-c5071e7bd3e4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.292338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d89f7-d2a0-48bc-b9ea-c5071e7bd3e4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.292620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d89f7-d2a0-48bc-b9ea-c5071e7bd3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.313292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-012d89f7-d2a0-48bc-b9ea-c5071e7bd3e4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:12.314952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-012d89f7-d2a0-48bc-b9ea-c5071e7bd3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:12.335511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb5f16-b344-4f5a-9111-52fdbdcd5e3c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.387250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5bf6c-0d52-4e9f-a60e-60464667851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.387997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5bf6c-0d52-4e9f-a60e-60464667851d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.388224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5bf6c-0d52-4e9f-a60e-60464667851d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.390305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5bf6c-0d52-4e9f-a60e-60464667851d None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.390305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5bf6c-0d52-4e9f-a60e-60464667851d None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:12.390305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5bf6c-0d52-4e9f-a60e-60464667851d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:12.391375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5bf6c-0d52-4e9f-a60e-60464667851d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:12.398456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 175/364] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:12.399351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d89f7-d2a0-48bc-b9ea-c5071e7bd3e4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.405351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 190/365] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:12.411053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5bf6c-0d52-4e9f-a60e-60464667851d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.447585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:12.516961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d89f7-d2a0-48bc-b9ea-c5071e7bd3e4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.526936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:12.530481 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5bf6c-0d52-4e9f-a60e-60464667851d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.558686 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2ec25-d729-487b-9e80-014db6dce116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.559628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2ec25-d729-487b-9e80-014db6dce116 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.559912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2ec25-d729-487b-9e80-014db6dce116 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.560620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2ec25-d729-487b-9e80-014db6dce116 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.561397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2ec25-d729-487b-9e80-014db6dce116 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:12.561981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2ec25-d729-487b-9e80-014db6dce116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:12.564110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2ec25-d729-487b-9e80-014db6dce116 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:12.574875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fd19d-b635-4ad3-bcc4-7e4fc60257d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.575524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 176/366] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:12.575760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fd19d-b635-4ad3-bcc4-7e4fc60257d4 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.576140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fd19d-b635-4ad3-bcc4-7e4fc60257d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.576250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2ec25-d729-487b-9e80-014db6dce116 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.576597 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fd19d-b635-4ad3-bcc4-7e4fc60257d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.672815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fd19d-b635-4ad3-bcc4-7e4fc60257d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:12.674720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fd19d-b635-4ad3-bcc4-7e4fc60257d4 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:12.694931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2ec25-d729-487b-9e80-014db6dce116 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.708508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:12.739994 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3214b5f-abd2-43cd-9504-2d9b71a6dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.741144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3214b5f-abd2-43cd-9504-2d9b71a6dbd1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.741651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3214b5f-abd2-43cd-9504-2d9b71a6dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.742121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3214b5f-abd2-43cd-9504-2d9b71a6dbd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.754454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3214b5f-abd2-43cd-9504-2d9b71a6dbd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-d3214b5f-abd2-43cd-9504-2d9b71a6dbd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:12.763885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:12.764841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:12.765657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:12.766274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 177/367] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:12.766849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3214b5f-abd2-43cd-9504-2d9b71a6dbd1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.806957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:12.815949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 191/368] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:40:12.821469 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fd19d-b635-4ad3-bcc4-7e4fc60257d4 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.900463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3214b5f-abd2-43cd-9504-2d9b71a6dbd1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.945602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b7c73-fe20-44ea-add9-332fa4ea0c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.946682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7c73-fe20-44ea-add9-332fa4ea0c10 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.947237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7c73-fe20-44ea-add9-332fa4ea0c10 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.947687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7c73-fe20-44ea-add9-332fa4ea0c10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.958669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fd19d-b635-4ad3-bcc4-7e4fc60257d4 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.988843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.004634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402496e-0070-42cb-a808-8dd05df478a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.005585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402496e-0070-42cb-a808-8dd05df478a7 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.005875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402496e-0070-42cb-a808-8dd05df478a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.006180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402496e-0070-42cb-a808-8dd05df478a7 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.006971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402496e-0070-42cb-a808-8dd05df478a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:13.007608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402496e-0070-42cb-a808-8dd05df478a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.009002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402496e-0070-42cb-a808-8dd05df478a7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.023295 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 192/369] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:13.024201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402496e-0070-42cb-a808-8dd05df478a7 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.031002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b7c73-fe20-44ea-add9-332fa4ea0c10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.032717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b7c73-fe20-44ea-add9-332fa4ea0c10 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.063756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.149050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402496e-0070-42cb-a808-8dd05df478a7 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.155644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 178/370] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:13.156803 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b7c73-fe20-44ea-add9-332fa4ea0c10 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.193924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea296a-a15e-4189-ba13-94e624f67777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.193924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea296a-a15e-4189-ba13-94e624f67777 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.193924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea296a-a15e-4189-ba13-94e624f67777 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.193924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea296a-a15e-4189-ba13-94e624f67777 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.199905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea296a-a15e-4189-ba13-94e624f67777 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.201375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea296a-a15e-4189-ba13-94e624f67777 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.203160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 193/371] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:13.204138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea296a-a15e-4189-ba13-94e624f67777 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.268834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.273566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b7c73-fe20-44ea-add9-332fa4ea0c10 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.301456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.311226 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4a861-8bbf-429a-8c40-9515860b9758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.312262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4a861-8bbf-429a-8c40-9515860b9758 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.312751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4a861-8bbf-429a-8c40-9515860b9758 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.313311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4a861-8bbf-429a-8c40-9515860b9758 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.320100 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea296a-a15e-4189-ba13-94e624f67777 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.358509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd096f-d3ed-43fd-b2f6-889d4e792157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.359389 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd096f-d3ed-43fd-b2f6-889d4e792157 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.359668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd096f-d3ed-43fd-b2f6-889d4e792157 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.359942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd096f-d3ed-43fd-b2f6-889d4e792157 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.361532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd096f-d3ed-43fd-b2f6-889d4e792157 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.363014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd096f-d3ed-43fd-b2f6-889d4e792157 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.364677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 194/372] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:13.365518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd096f-d3ed-43fd-b2f6-889d4e792157 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.393820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4a861-8bbf-429a-8c40-9515860b9758 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.395691 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4a861-8bbf-429a-8c40-9515860b9758 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.470033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd096f-d3ed-43fd-b2f6-889d4e792157 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.494988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.504051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 179/373] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:13.505378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4a861-8bbf-429a-8c40-9515860b9758 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.507874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0bc27-5a9b-44f7-b68f-8267b1aceb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.508831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0bc27-5a9b-44f7-b68f-8267b1aceb0d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.509106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0bc27-5a9b-44f7-b68f-8267b1aceb0d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.509374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0bc27-5a9b-44f7-b68f-8267b1aceb0d None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.510835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0bc27-5a9b-44f7-b68f-8267b1aceb0d None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.512597 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0bc27-5a9b-44f7-b68f-8267b1aceb0d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.514816 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 195/374] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:13.515691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0bc27-5a9b-44f7-b68f-8267b1aceb0d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.538559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.615857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4a861-8bbf-429a-8c40-9515860b9758 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.626051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0bc27-5a9b-44f7-b68f-8267b1aceb0d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.653320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eb5f5-1a8c-4a96-a853-f92dadefacfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.654646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb5f5-1a8c-4a96-a853-f92dadefacfa None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.655160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb5f5-1a8c-4a96-a853-f92dadefacfa None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.655735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb5f5-1a8c-4a96-a853-f92dadefacfa None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.657241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497eb5f5-1a8c-4a96-a853-f92dadefacfa None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.659141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497eb5f5-1a8c-4a96-a853-f92dadefacfa None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.661166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 180/375] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:13.661527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126f737-8166-45b4-9288-a39baeb5d602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.662382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eb5f5-1a8c-4a96-a853-f92dadefacfa None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.662534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126f737-8166-45b4-9288-a39baeb5d602 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.662766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126f737-8166-45b4-9288-a39baeb5d602 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.663041 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126f737-8166-45b4-9288-a39baeb5d602 None admin] PATH_INFO: `/v3/projects/799d47101892496a8efa7ee2b0f5f24a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.664240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126f737-8166-45b4-9288-a39baeb5d602 None admin] RBAC: Authorizing `identity:get_project(project_id=799d47101892496a8efa7ee2b0f5f24a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.667512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126f737-8166-45b4-9288-a39baeb5d602 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.669560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 196/376] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/projects/799d47101892496a8efa7ee2b0f5f24a => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:13.670389 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126f737-8166-45b4-9288-a39baeb5d602 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.732876 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.768524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.775358 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eb5f5-1a8c-4a96-a853-f92dadefacfa None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.775503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126f737-8166-45b4-9288-a39baeb5d602 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.810767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277aa1a-880c-4994-829b-86e2ccc11530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.815709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277aa1a-880c-4994-829b-86e2ccc11530 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.815709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277aa1a-880c-4994-829b-86e2ccc11530 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.815709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277aa1a-880c-4994-829b-86e2ccc11530 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.815709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277aa1a-880c-4994-829b-86e2ccc11530 None admin] RBAC: Adding query filter params (group.id=e902269caaf94142a68bebe80b795809, role.id=73a019c7fb8340a58c7dfe4550341949, scope.project.id=14afae0ea8a043bca3106a84957c35a0) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:13.815709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277aa1a-880c-4994-829b-86e2ccc11530 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.815709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277aa1a-880c-4994-829b-86e2ccc11530 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.818268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00582597-f581-4ad4-bef0-371e26a255ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.818948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00582597-f581-4ad4-bef0-371e26a255ea None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.819223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00582597-f581-4ad4-bef0-371e26a255ea None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.819496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00582597-f581-4ad4-bef0-371e26a255ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.820349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00582597-f581-4ad4-bef0-371e26a255ea None admin] RBAC: Adding query filter params (group.id=e902269caaf94142a68bebe80b795809, role.id=5dce8d92cffa40f79dfa1c03724c47e2, scope.project.id=14afae0ea8a043bca3106a84957c35a0) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:13.820652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00582597-f581-4ad4-bef0-371e26a255ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.822056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00582597-f581-4ad4-bef0-371e26a255ea None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.829107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 197/377] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/role_assignments?group.id=e902269caaf94142a68bebe80b795809&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=73a019c7fb8340a58c7dfe4550341949 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:13.830266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277aa1a-880c-4994-829b-86e2ccc11530 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.830634 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 181/378] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/role_assignments?group.id=e902269caaf94142a68bebe80b795809&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:13.960007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277aa1a-880c-4994-829b-86e2ccc11530 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.987397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:14.001739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608168bc-974a-4cb9-9d91-33d4f387629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:14.006640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608168bc-974a-4cb9-9d91-33d4f387629a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:14.006640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608168bc-974a-4cb9-9d91-33d4f387629a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:14.006640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608168bc-974a-4cb9-9d91-33d4f387629a None admin] PATH_INFO: `/v3/groups/3f78e93462734440a78315b00ae273be` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:14.011226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608168bc-974a-4cb9-9d91-33d4f387629a None admin] RBAC: Authorizing `identity:get_group(group_id=3f78e93462734440a78315b00ae273be)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:14.013874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608168bc-974a-4cb9-9d91-33d4f387629a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:14.016391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 198/379] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/groups/3f78e93462734440a78315b00ae273be => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:14.022047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00582597-f581-4ad4-bef0-371e26a255ea None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:14.133569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00582597-f581-4ad4-bef0-371e26a255ea None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:14.176148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5a3bc-df78-4191-80ae-7af37eba33f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:14.177247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5a3bc-df78-4191-80ae-7af37eba33f6 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:14.177786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5a3bc-df78-4191-80ae-7af37eba33f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:14.178269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5a3bc-df78-4191-80ae-7af37eba33f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:14.179381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5a3bc-df78-4191-80ae-7af37eba33f6 None admin] RBAC: Adding query filter params (group.id=3f78e93462734440a78315b00ae273be, role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=799d47101892496a8efa7ee2b0f5f24a) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:14.179926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5a3bc-df78-4191-80ae-7af37eba33f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:14.181669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5a3bc-df78-4191-80ae-7af37eba33f6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:14.192997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 182/380] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 05:40:14 2022] GET /identity/v3/role_assignments?group.id=3f78e93462734440a78315b00ae273be&scope.project.id=799d47101892496a8efa7ee2b0f5f24a&role.id=7a833c45603a4830b888e58e1232712a => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:14.424010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f837d-b1ee-4c4b-97d3-1f20ca9a2b99 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:14.424728 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f837d-b1ee-4c4b-97d3-1f20ca9a2b99 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:14.425224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f837d-b1ee-4c4b-97d3-1f20ca9a2b99 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:14.427224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 199/381] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:14.432784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc316a-3c9b-440d-b1b9-469ad6389964 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:14.433448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc316a-3c9b-440d-b1b9-469ad6389964 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:14.434435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc316a-3c9b-440d-b1b9-469ad6389964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:14.455842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-16cc316a-3c9b-440d-b1b9-469ad6389964 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:14.457402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-16cc316a-3c9b-440d-b1b9-469ad6389964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:14.481201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:14.551054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 183/382] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:14.591078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f837d-b1ee-4c4b-97d3-1f20ca9a2b99 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:14.731609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f837d-b1ee-4c4b-97d3-1f20ca9a2b99 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:14.773535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384a8c3-082d-4bd9-90c6-a124081bafd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:14.775189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a8c3-082d-4bd9-90c6-a124081bafd0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:14.775742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a8c3-082d-4bd9-90c6-a124081bafd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:14.776245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a8c3-082d-4bd9-90c6-a124081bafd0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:14.793379 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384a8c3-082d-4bd9-90c6-a124081bafd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-4384a8c3-082d-4bd9-90c6-a124081bafd0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:14.801797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:14.802773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:14.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:14.805984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 200/383] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:14.812590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc316a-3c9b-440d-b1b9-469ad6389964 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:14.836310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64469e50-5934-46de-a044-e9b43e2d827d None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:14.837081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64469e50-5934-46de-a044-e9b43e2d827d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:14.837580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64469e50-5934-46de-a044-e9b43e2d827d None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:14.839232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 201/384] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:14.846404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ef32-add6-4168-bfde-0004b48c9090 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:14.846999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ef32-add6-4168-bfde-0004b48c9090 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:14.847518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ef32-add6-4168-bfde-0004b48c9090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:14.868557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e580ef32-add6-4168-bfde-0004b48c9090 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:14.870338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e580ef32-add6-4168-bfde-0004b48c9090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:14.919282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:14.947702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 202/385] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:14.975087 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580ef32-add6-4168-bfde-0004b48c9090 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:15.034324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc316a-3c9b-440d-b1b9-469ad6389964 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:15.101070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580ef32-add6-4168-bfde-0004b48c9090 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:15.124879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a6b32-a799-4a16-8b12-f0b941d42c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:15.125569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a6b32-a799-4a16-8b12-f0b941d42c6d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.125856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a6b32-a799-4a16-8b12-f0b941d42c6d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.126135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a6b32-a799-4a16-8b12-f0b941d42c6d None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.126916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a6b32-a799-4a16-8b12-f0b941d42c6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:15.127228 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a6b32-a799-4a16-8b12-f0b941d42c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:15.128599 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a6b32-a799-4a16-8b12-f0b941d42c6d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:15.139149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed16d10-5fc1-4509-88d2-4f0af7a68038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:15.140016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed16d10-5fc1-4509-88d2-4f0af7a68038 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.140306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed16d10-5fc1-4509-88d2-4f0af7a68038 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.140633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed16d10-5fc1-4509-88d2-4f0af7a68038 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.146624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 184/386] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:15.152100 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed16d10-5fc1-4509-88d2-4f0af7a68038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:15.153499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a6b32-a799-4a16-8b12-f0b941d42c6d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-aed16d10-5fc1-4509-88d2-4f0af7a68038 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:15.160834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:15.161706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:15.162946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 203/387] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:15.168042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed16d10-5fc1-4509-88d2-4f0af7a68038 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:15.202919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:15.207825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:15.278893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed16d10-5fc1-4509-88d2-4f0af7a68038 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:15.322953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131d891-c3c3-4af5-93b0-be3a72ab91d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:15.323868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131d891-c3c3-4af5-93b0-be3a72ab91d0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.324851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131d891-c3c3-4af5-93b0-be3a72ab91d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.325281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131d891-c3c3-4af5-93b0-be3a72ab91d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.326200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131d891-c3c3-4af5-93b0-be3a72ab91d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:15.326674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131d891-c3c3-4af5-93b0-be3a72ab91d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:15.328230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131d891-c3c3-4af5-93b0-be3a72ab91d0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:15.328610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a6b32-a799-4a16-8b12-f0b941d42c6d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:15.338217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 204/388] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:15.344336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131d891-c3c3-4af5-93b0-be3a72ab91d0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:15.391200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbde51-06b9-4196-bb13-d55f580113b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:15.392453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbde51-06b9-4196-bb13-d55f580113b4 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.392590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbde51-06b9-4196-bb13-d55f580113b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.392948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbde51-06b9-4196-bb13-d55f580113b4 None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.394502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbde51-06b9-4196-bb13-d55f580113b4 None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:15.395876 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbde51-06b9-4196-bb13-d55f580113b4 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:15.399223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 185/389] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:15 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:15.404096 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbde51-06b9-4196-bb13-d55f580113b4 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:15.454756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:15.459404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131d891-c3c3-4af5-93b0-be3a72ab91d0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:15.494991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da5c02-2bb8-4101-9b6b-3ce85020bab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:15.496677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da5c02-2bb8-4101-9b6b-3ce85020bab4 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.497103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da5c02-2bb8-4101-9b6b-3ce85020bab4 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.497567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da5c02-2bb8-4101-9b6b-3ce85020bab4 None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.498942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da5c02-2bb8-4101-9b6b-3ce85020bab4 None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:15.500611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da5c02-2bb8-4101-9b6b-3ce85020bab4 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:15.505736 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 205/390] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:15 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:15.508272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da5c02-2bb8-4101-9b6b-3ce85020bab4 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:15.570080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:15.600004 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbde51-06b9-4196-bb13-d55f580113b4 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:15.614495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da5c02-2bb8-4101-9b6b-3ce85020bab4 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:15.651197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e396e-4a19-4b6d-9e60-c69a21797bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:15.652765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e396e-4a19-4b6d-9e60-c69a21797bac None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.653262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e396e-4a19-4b6d-9e60-c69a21797bac None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.653737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e396e-4a19-4b6d-9e60-c69a21797bac None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.655747 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e396e-4a19-4b6d-9e60-c69a21797bac None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:15.657234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e396e-4a19-4b6d-9e60-c69a21797bac None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:15.659411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 206/391] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:15 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:15.667331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e396e-4a19-4b6d-9e60-c69a21797bac None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:15.668343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6515b9-2c2a-48b8-ad03-0e35c19d2607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:15.669457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6515b9-2c2a-48b8-ad03-0e35c19d2607 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.669943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6515b9-2c2a-48b8-ad03-0e35c19d2607 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.670418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6515b9-2c2a-48b8-ad03-0e35c19d2607 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.674274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6515b9-2c2a-48b8-ad03-0e35c19d2607 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:15.675971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6515b9-2c2a-48b8-ad03-0e35c19d2607 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:15.678224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 186/392] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:15 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:15.684583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6515b9-2c2a-48b8-ad03-0e35c19d2607 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:15.712315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:15.782488 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e396e-4a19-4b6d-9e60-c69a21797bac None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:15.819281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30274553-e39c-4262-ba07-6bb5e817a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:15.820172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30274553-e39c-4262-ba07-6bb5e817a953 None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.820453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30274553-e39c-4262-ba07-6bb5e817a953 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.820727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30274553-e39c-4262-ba07-6bb5e817a953 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271/groups/e902269caaf94142a68bebe80b795809/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.822771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30274553-e39c-4262-ba07-6bb5e817a953 None admin] RBAC: Authorizing `identity:create_grant(project_id=56fabc309a30484ebda9a05789fa5271, group_id=e902269caaf94142a68bebe80b795809, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:15.824263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30274553-e39c-4262-ba07-6bb5e817a953 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:15.864985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6515b9-2c2a-48b8-ad03-0e35c19d2607 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:15.904892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 207/393] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Jul 25 05:40:15 2022] PUT /identity/v3/projects/56fabc309a30484ebda9a05789fa5271/groups/e902269caaf94142a68bebe80b795809/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:40:15.909544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2bde5-fa40-4006-b02c-716fc5ecfad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:15.911063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2bde5-fa40-4006-b02c-716fc5ecfad4 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.911063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2bde5-fa40-4006-b02c-716fc5ecfad4 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.911207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2bde5-fa40-4006-b02c-716fc5ecfad4 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271/groups/e902269caaf94142a68bebe80b795809/roles/73a019c7fb8340a58c7dfe4550341949` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.913732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2bde5-fa40-4006-b02c-716fc5ecfad4 None admin] RBAC: Authorizing `identity:create_grant(project_id=56fabc309a30484ebda9a05789fa5271, group_id=e902269caaf94142a68bebe80b795809, role_id=73a019c7fb8340a58c7dfe4550341949)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:15.915313 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2bde5-fa40-4006-b02c-716fc5ecfad4 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:16.004875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 187/394] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Jul 25 05:40:15 2022] PUT /identity/v3/projects/56fabc309a30484ebda9a05789fa5271/groups/e902269caaf94142a68bebe80b795809/roles/73a019c7fb8340a58c7dfe4550341949 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:16.434010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3258d9-d6a3-4d15-9705-8aa89bce63d9 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.434727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3258d9-d6a3-4d15-9705-8aa89bce63d9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.435217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3258d9-d6a3-4d15-9705-8aa89bce63d9 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.437097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 208/395] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:16.442147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011c46e-f42a-4962-9caa-85b1ad9cf8fe None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.442514 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011c46e-f42a-4962-9caa-85b1ad9cf8fe None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.442847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011c46e-f42a-4962-9caa-85b1ad9cf8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.444420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c949-f57f-4942-902a-ca1bd9bc8a45 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.444920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c949-f57f-4942-902a-ca1bd9bc8a45 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.445194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c949-f57f-4942-902a-ca1bd9bc8a45 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.446671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 209/396] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:16.454247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0efd9-4518-4122-b0c0-d8eed529c2c6 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.454247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0efd9-4518-4122-b0c0-d8eed529c2c6 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.454247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0efd9-4518-4122-b0c0-d8eed529c2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.470840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c011c46e-f42a-4962-9caa-85b1ad9cf8fe None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:16.472120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c011c46e-f42a-4962-9caa-85b1ad9cf8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:16.473369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9cc0efd9-4518-4122-b0c0-d8eed529c2c6 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:16.474801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9cc0efd9-4518-4122-b0c0-d8eed529c2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:16.481125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:16.581911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 188/397] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:16.606629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 210/398] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:16.616385 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011c46e-f42a-4962-9caa-85b1ad9cf8fe None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:16.644412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0efd9-4518-4122-b0c0-d8eed529c2c6 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:16.695644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:16.741460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011c46e-f42a-4962-9caa-85b1ad9cf8fe None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:16.805314 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607467cc-a082-4c12-90e8-100d776b5d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:16.806650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607467cc-a082-4c12-90e8-100d776b5d44 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.807157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607467cc-a082-4c12-90e8-100d776b5d44 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.807646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607467cc-a082-4c12-90e8-100d776b5d44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.821347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607467cc-a082-4c12-90e8-100d776b5d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-607467cc-a082-4c12-90e8-100d776b5d44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:16.832001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:16.832930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:16.833757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:16.836071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 189/399] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:16.841543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607467cc-a082-4c12-90e8-100d776b5d44 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:16.847742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:16.868390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0efd9-4518-4122-b0c0-d8eed529c2c6 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:16.932508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425b480-6a65-4618-9eb5-220862fa26ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:16.933849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425b480-6a65-4618-9eb5-220862fa26ec None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.934135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425b480-6a65-4618-9eb5-220862fa26ec None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.934407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425b480-6a65-4618-9eb5-220862fa26ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.950705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425b480-6a65-4618-9eb5-220862fa26ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5425b480-6a65-4618-9eb5-220862fa26ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:16.963451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:16.964382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:16.965643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 211/400] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:16.971607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425b480-6a65-4618-9eb5-220862fa26ec None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.027877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607467cc-a082-4c12-90e8-100d776b5d44 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.087767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c6fd1-545f-4d2b-bc9a-ab8227c14a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.088440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c6fd1-545f-4d2b-bc9a-ab8227c14a75 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.088722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c6fd1-545f-4d2b-bc9a-ab8227c14a75 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.088998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c6fd1-545f-4d2b-bc9a-ab8227c14a75 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.089778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c6fd1-545f-4d2b-bc9a-ab8227c14a75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:17.090085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c6fd1-545f-4d2b-bc9a-ab8227c14a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.091484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c6fd1-545f-4d2b-bc9a-ab8227c14a75 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.108432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 190/401] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:17.113796 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c6fd1-545f-4d2b-bc9a-ab8227c14a75 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.115804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.146634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425b480-6a65-4618-9eb5-220862fa26ec None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.193069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92dd8b-fdac-4b78-8ae7-7159c48e7386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.194057 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92dd8b-fdac-4b78-8ae7-7159c48e7386 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.194636 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92dd8b-fdac-4b78-8ae7-7159c48e7386 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.194924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92dd8b-fdac-4b78-8ae7-7159c48e7386 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.195827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92dd8b-fdac-4b78-8ae7-7159c48e7386 None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:17.196141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92dd8b-fdac-4b78-8ae7-7159c48e7386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.198145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92dd8b-fdac-4b78-8ae7-7159c48e7386 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.208620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 212/402] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:17.210273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cba3f-63ee-490e-860c-f4eb41f0e5ef None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.210586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cba3f-63ee-490e-860c-f4eb41f0e5ef None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.210865 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cba3f-63ee-490e-860c-f4eb41f0e5ef None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.212612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 213/403] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:17.215208 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.217265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cba3f-63ee-490e-860c-f4eb41f0e5ef None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.238068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c6fd1-545f-4d2b-bc9a-ab8227c14a75 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.275375 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca343a-6821-47df-9677-c0158aeecf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.276383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca343a-6821-47df-9677-c0158aeecf27 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.276727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca343a-6821-47df-9677-c0158aeecf27 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.277097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca343a-6821-47df-9677-c0158aeecf27 None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.278346 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca343a-6821-47df-9677-c0158aeecf27 None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.279879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca343a-6821-47df-9677-c0158aeecf27 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.281539 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 191/404] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:17 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:17.282910 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d846-189c-46c0-8b3b-3f6e9f1aca41 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.283198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d846-189c-46c0-8b3b-3f6e9f1aca41 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.283470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d846-189c-46c0-8b3b-3f6e9f1aca41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.302919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c2a6d846-189c-46c0-8b3b-3f6e9f1aca41 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:17.304211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c2a6d846-189c-46c0-8b3b-3f6e9f1aca41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:17.351098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cba3f-63ee-490e-860c-f4eb41f0e5ef None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.379987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 192/405] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:17.381063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6d846-189c-46c0-8b3b-3f6e9f1aca41 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.388931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc7feb-804b-49cc-9097-b7713e8d27e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.390043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7feb-804b-49cc-9097-b7713e8d27e9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.390496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7feb-804b-49cc-9097-b7713e8d27e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.391002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7feb-804b-49cc-9097-b7713e8d27e9 None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.392422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc7feb-804b-49cc-9097-b7713e8d27e9 None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.394120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc7feb-804b-49cc-9097-b7713e8d27e9 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.400512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 214/406] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:17 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:17.401689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc7feb-804b-49cc-9097-b7713e8d27e9 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.418907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.454895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.492637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6d846-189c-46c0-8b3b-3f6e9f1aca41 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.514231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc7feb-804b-49cc-9097-b7713e8d27e9 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.531235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27ec78-57d4-4f61-b578-c7afe21c15a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.532216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27ec78-57d4-4f61-b578-c7afe21c15a9 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.532567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27ec78-57d4-4f61-b578-c7afe21c15a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.533008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27ec78-57d4-4f61-b578-c7afe21c15a9 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.534361 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27ec78-57d4-4f61-b578-c7afe21c15a9 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.536266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27ec78-57d4-4f61-b578-c7afe21c15a9 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.538634 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 193/407] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:17 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:17.539569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27ec78-57d4-4f61-b578-c7afe21c15a9 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.553841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19a847-5016-4fac-9927-423b2af3a9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.555010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19a847-5016-4fac-9927-423b2af3a9a6 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.555472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19a847-5016-4fac-9927-423b2af3a9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.556029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19a847-5016-4fac-9927-423b2af3a9a6 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.557501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19a847-5016-4fac-9927-423b2af3a9a6 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.559721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19a847-5016-4fac-9927-423b2af3a9a6 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.566793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 215/408] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:17 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:17.567953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19a847-5016-4fac-9927-423b2af3a9a6 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.652859 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27ec78-57d4-4f61-b578-c7afe21c15a9 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.656067 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.675126 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19a847-5016-4fac-9927-423b2af3a9a6 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.683063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.689448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1bade-d31a-4e88-8e88-1b0a0c0d01c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.690317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1bade-d31a-4e88-8e88-1b0a0c0d01c8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.690694 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1bade-d31a-4e88-8e88-1b0a0c0d01c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.691030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1bade-d31a-4e88-8e88-1b0a0c0d01c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.703525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1bade-d31a-4e88-8e88-1b0a0c0d01c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-96f1bade-d31a-4e88-8e88-1b0a0c0d01c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:17.712789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:17.713696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:17.714579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 194/409] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:17.715290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1bade-d31a-4e88-8e88-1b0a0c0d01c8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.718175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7eded-56fe-42a8-8690-ab7876bd868f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.719383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7eded-56fe-42a8-8690-ab7876bd868f None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.719878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7eded-56fe-42a8-8690-ab7876bd868f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.720356 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7eded-56fe-42a8-8690-ab7876bd868f None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0/groups/e902269caaf94142a68bebe80b795809/roles/73a019c7fb8340a58c7dfe4550341949` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.722894 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7eded-56fe-42a8-8690-ab7876bd868f None admin] RBAC: Authorizing `identity:create_grant(project_id=14afae0ea8a043bca3106a84957c35a0, group_id=e902269caaf94142a68bebe80b795809, role_id=73a019c7fb8340a58c7dfe4550341949)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.724562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7eded-56fe-42a8-8690-ab7876bd868f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.819445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1bade-d31a-4e88-8e88-1b0a0c0d01c8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.826658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 216/410] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Jul 25 05:40:17 2022] PUT /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0/groups/e902269caaf94142a68bebe80b795809/roles/73a019c7fb8340a58c7dfe4550341949 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:17.828015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7eded-56fe-42a8-8690-ab7876bd868f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.860680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fc46a-94c2-4b88-9441-463a0d5d6a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.861592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fc46a-94c2-4b88-9441-463a0d5d6a7c None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.861869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fc46a-94c2-4b88-9441-463a0d5d6a7c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.862167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fc46a-94c2-4b88-9441-463a0d5d6a7c None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0/groups/e902269caaf94142a68bebe80b795809/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.868879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fc46a-94c2-4b88-9441-463a0d5d6a7c None admin] RBAC: Authorizing `identity:create_grant(project_id=14afae0ea8a043bca3106a84957c35a0, group_id=e902269caaf94142a68bebe80b795809, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.870371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fc46a-94c2-4b88-9441-463a0d5d6a7c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.971149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 195/411] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Jul 25 05:40:17 2022] PUT /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0/groups/e902269caaf94142a68bebe80b795809/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:17.995485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7eded-56fe-42a8-8690-ab7876bd868f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:18.036772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99e54c-5c68-4388-8972-a206d6c45740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:18.037466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99e54c-5c68-4388-8972-a206d6c45740 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:18.037811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99e54c-5c68-4388-8972-a206d6c45740 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:18.038025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99e54c-5c68-4388-8972-a206d6c45740 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:18.038868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99e54c-5c68-4388-8972-a206d6c45740 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:18.039180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99e54c-5c68-4388-8972-a206d6c45740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:18.040580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99e54c-5c68-4388-8972-a206d6c45740 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:18.050974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 217/412] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:18.056180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fc46a-94c2-4b88-9441-463a0d5d6a7c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:18.119397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:18.177683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fc46a-94c2-4b88-9441-463a0d5d6a7c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:18.224115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2e1f1-e101-436b-81aa-8870dfcb4187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:18.224988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2e1f1-e101-436b-81aa-8870dfcb4187 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:18.225271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2e1f1-e101-436b-81aa-8870dfcb4187 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:18.225551 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2e1f1-e101-436b-81aa-8870dfcb4187 None admin] PATH_INFO: `/v3/groups/3f78e93462734440a78315b00ae273be` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:18.231204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2e1f1-e101-436b-81aa-8870dfcb4187 None admin] RBAC: Authorizing `identity:get_group(group_id=3f78e93462734440a78315b00ae273be)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:18.232719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2e1f1-e101-436b-81aa-8870dfcb4187 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:18.234613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 196/413] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:18 2022] GET /identity/v3/groups/3f78e93462734440a78315b00ae273be => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:18.240542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99e54c-5c68-4388-8972-a206d6c45740 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:18.250049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:18.377010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99e54c-5c68-4388-8972-a206d6c45740 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:18.411674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd77716-e68e-4448-bd52-b8bdbd7dc4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:18.412544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd77716-e68e-4448-bd52-b8bdbd7dc4bc None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:18.412819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd77716-e68e-4448-bd52-b8bdbd7dc4bc None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:18.413088 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd77716-e68e-4448-bd52-b8bdbd7dc4bc None admin] PATH_INFO: `/v3/projects/799d47101892496a8efa7ee2b0f5f24a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:18.414288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd77716-e68e-4448-bd52-b8bdbd7dc4bc None admin] RBAC: Authorizing `identity:get_project(project_id=799d47101892496a8efa7ee2b0f5f24a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:18.415945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd77716-e68e-4448-bd52-b8bdbd7dc4bc None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:18.418587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 218/414] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:18 2022] GET /identity/v3/projects/799d47101892496a8efa7ee2b0f5f24a => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:18.427675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2e1f1-e101-436b-81aa-8870dfcb4187 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:18.450756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd723-787c-4bde-8184-ba7d574b3073 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:18.450939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd723-787c-4bde-8184-ba7d574b3073 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:18.451175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd723-787c-4bde-8184-ba7d574b3073 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:18.452837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 219/415] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:18.457769 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669691dd-f243-42ee-a2c0-e31d77160a9a None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:18.458030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669691dd-f243-42ee-a2c0-e31d77160a9a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:18.458326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669691dd-f243-42ee-a2c0-e31d77160a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:18.482289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-669691dd-f243-42ee-a2c0-e31d77160a9a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:18.483754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-669691dd-f243-42ee-a2c0-e31d77160a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:18.561785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 220/416] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:18.585319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669691dd-f243-42ee-a2c0-e31d77160a9a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:18.587574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:18.623110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2e1f1-e101-436b-81aa-8870dfcb4187 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:18.643822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:18.662064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d79d2-0757-4728-aa81-d72f5eaecbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:18.663043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d79d2-0757-4728-aa81-d72f5eaecbb6 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:18.663327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d79d2-0757-4728-aa81-d72f5eaecbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:18.666291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d79d2-0757-4728-aa81-d72f5eaecbb6 None admin] PATH_INFO: `/v3/projects/799d47101892496a8efa7ee2b0f5f24a/groups/3f78e93462734440a78315b00ae273be/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:18.666291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d79d2-0757-4728-aa81-d72f5eaecbb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=799d47101892496a8efa7ee2b0f5f24a, group_id=3f78e93462734440a78315b00ae273be, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:18.667119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d79d2-0757-4728-aa81-d72f5eaecbb6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:18.694918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669691dd-f243-42ee-a2c0-e31d77160a9a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:18.730846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db17ea-9c60-4cfc-b627-bbd8140b5ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:18.731658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db17ea-9c60-4cfc-b627-bbd8140b5ec7 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:18.731932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db17ea-9c60-4cfc-b627-bbd8140b5ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:18.732228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db17ea-9c60-4cfc-b627-bbd8140b5ec7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:18.742483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db17ea-9c60-4cfc-b627-bbd8140b5ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-c2db17ea-9c60-4cfc-b627-bbd8140b5ec7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:18.750539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:18.751531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:18.752392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 221/417] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:18.753101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6328fc-6c1e-415f-aed6-9b38d6145d07 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:18.758603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6328fc-6c1e-415f-aed6-9b38d6145d07 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:18.758603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6328fc-6c1e-415f-aed6-9b38d6145d07 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:18.758791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 222/418] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:18.759483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6328fc-6c1e-415f-aed6-9b38d6145d07 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:18.783156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 197/419] 10.222.0.17 () {58 vars in 1442 bytes} [Mon Jul 25 05:40:18 2022] PUT /identity/v3/projects/799d47101892496a8efa7ee2b0f5f24a/groups/3f78e93462734440a78315b00ae273be/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:18.785344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808023b-501b-4a7c-9150-0053c9a1309e None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:18.791719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808023b-501b-4a7c-9150-0053c9a1309e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:18.791719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808023b-501b-4a7c-9150-0053c9a1309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:18.828550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0808023b-501b-4a7c-9150-0053c9a1309e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:18.829774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0808023b-501b-4a7c-9150-0053c9a1309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:18.877982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:18.935340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6328fc-6c1e-415f-aed6-9b38d6145d07 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:18.972764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732845-8210-4ee5-91fd-1194cdae20df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:18.973541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732845-8210-4ee5-91fd-1194cdae20df None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:18.973928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732845-8210-4ee5-91fd-1194cdae20df None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:18.974208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732845-8210-4ee5-91fd-1194cdae20df None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:18.974991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732845-8210-4ee5-91fd-1194cdae20df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:18.975293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732845-8210-4ee5-91fd-1194cdae20df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:18.976796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732845-8210-4ee5-91fd-1194cdae20df None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:18.982439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 198/420] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:18.985939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 223/421] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:18.991391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808023b-501b-4a7c-9150-0053c9a1309e None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:19.011115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732845-8210-4ee5-91fd-1194cdae20df None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:19.103958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808023b-501b-4a7c-9150-0053c9a1309e None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:19.114991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:19.120101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732845-8210-4ee5-91fd-1194cdae20df None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:19.142925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6002f2f-a988-4bf6-b2c1-63aade62557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:19.143879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6002f2f-a988-4bf6-b2c1-63aade62557c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:19.144165 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6002f2f-a988-4bf6-b2c1-63aade62557c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:19.144440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6002f2f-a988-4bf6-b2c1-63aade62557c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:19.145391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:19.158394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabedb42-e61f-43dc-b17c-7176ae7e73bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:19.164093 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabedb42-e61f-43dc-b17c-7176ae7e73bf None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:19.164093 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabedb42-e61f-43dc-b17c-7176ae7e73bf None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:19.164093 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabedb42-e61f-43dc-b17c-7176ae7e73bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:19.172439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabedb42-e61f-43dc-b17c-7176ae7e73bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-aabedb42-e61f-43dc-b17c-7176ae7e73bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:19.180439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:19.181506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:19.182499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:19.188812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 224/422] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:19.190860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabedb42-e61f-43dc-b17c-7176ae7e73bf None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:19.225920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6002f2f-a988-4bf6-b2c1-63aade62557c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:19.228332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6002f2f-a988-4bf6-b2c1-63aade62557c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:19.315146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabedb42-e61f-43dc-b17c-7176ae7e73bf None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:19.379875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8dbb6-da8b-4d2f-9d89-a115040d5022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:19.380546 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8dbb6-da8b-4d2f-9d89-a115040d5022 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:19.380825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8dbb6-da8b-4d2f-9d89-a115040d5022 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:19.381098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8dbb6-da8b-4d2f-9d89-a115040d5022 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:19.381849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8dbb6-da8b-4d2f-9d89-a115040d5022 None admin] RBAC: Adding query filter params (name=member) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:19.382154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8dbb6-da8b-4d2f-9d89-a115040d5022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:19.383570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8dbb6-da8b-4d2f-9d89-a115040d5022 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:19.398617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 199/423] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:40:19.400862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 225/424] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:19.408873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6002f2f-a988-4bf6-b2c1-63aade62557c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:19.410140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8dbb6-da8b-4d2f-9d89-a115040d5022 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:19.419754 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:19.515690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6002f2f-a988-4bf6-b2c1-63aade62557c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:19.518187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:19.565563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566b46f-1319-47a9-bbe9-72f3af1e62de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:19.571523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566b46f-1319-47a9-bbe9-72f3af1e62de None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:19.572031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566b46f-1319-47a9-bbe9-72f3af1e62de None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:19.572503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566b46f-1319-47a9-bbe9-72f3af1e62de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:19.619054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8dbb6-da8b-4d2f-9d89-a115040d5022 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:19.657180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54e4c7-d9f4-49e9-83b1-7cf86fa837f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:19.658390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54e4c7-d9f4-49e9-83b1-7cf86fa837f7 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:19.658911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54e4c7-d9f4-49e9-83b1-7cf86fa837f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:19.659380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54e4c7-d9f4-49e9-83b1-7cf86fa837f7 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:19.660776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54e4c7-d9f4-49e9-83b1-7cf86fa837f7 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:19.662432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54e4c7-d9f4-49e9-83b1-7cf86fa837f7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:19.664840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 226/425] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:19 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:19.670349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54e4c7-d9f4-49e9-83b1-7cf86fa837f7 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:19.774088 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54e4c7-d9f4-49e9-83b1-7cf86fa837f7 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:19.797411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:19.803316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566b46f-1319-47a9-bbe9-72f3af1e62de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:19.804797 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566b46f-1319-47a9-bbe9-72f3af1e62de None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:19.812284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25201a-26d6-4ec6-87b8-0b7b298d8ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:19.813414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25201a-26d6-4ec6-87b8-0b7b298d8ca8 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:19.816961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25201a-26d6-4ec6-87b8-0b7b298d8ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:19.816961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25201a-26d6-4ec6-87b8-0b7b298d8ca8 None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:19.816961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25201a-26d6-4ec6-87b8-0b7b298d8ca8 None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:19.817641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25201a-26d6-4ec6-87b8-0b7b298d8ca8 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:19.819876 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 227/426] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:19 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:19.825409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:19.826863 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25201a-26d6-4ec6-87b8-0b7b298d8ca8 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:19.932929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25201a-26d6-4ec6-87b8-0b7b298d8ca8 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:19.967325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd932ca-3a11-4e19-8f50-0b9590f275ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:19.968280 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd932ca-3a11-4e19-8f50-0b9590f275ee None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:19.968781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd932ca-3a11-4e19-8f50-0b9590f275ee None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:19.969226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd932ca-3a11-4e19-8f50-0b9590f275ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:19.970223 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd932ca-3a11-4e19-8f50-0b9590f275ee None admin] RBAC: Adding query filter params (group.id=e902269caaf94142a68bebe80b795809, role.id=73a019c7fb8340a58c7dfe4550341949, scope.project.id=56fabc309a30484ebda9a05789fa5271) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:19.970741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd932ca-3a11-4e19-8f50-0b9590f275ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:19.972367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd932ca-3a11-4e19-8f50-0b9590f275ee None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:19.982842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 228/427] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 05:40:19 2022] GET /identity/v3/role_assignments?group.id=e902269caaf94142a68bebe80b795809&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=73a019c7fb8340a58c7dfe4550341949 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:20.064479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 200/428] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:20.069998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd932ca-3a11-4e19-8f50-0b9590f275ee None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.139106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.194152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd932ca-3a11-4e19-8f50-0b9590f275ee None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.236826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d85b5-3132-4760-a20b-9332813d3f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.237922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d85b5-3132-4760-a20b-9332813d3f43 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.238365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d85b5-3132-4760-a20b-9332813d3f43 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.238803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d85b5-3132-4760-a20b-9332813d3f43 None admin] PATH_INFO: `/v3/projects/56fabc309a30484ebda9a05789fa5271` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.240128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d85b5-3132-4760-a20b-9332813d3f43 None admin] RBAC: Authorizing `identity:get_project(project_id=56fabc309a30484ebda9a05789fa5271)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:20.241728 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d85b5-3132-4760-a20b-9332813d3f43 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:20.243888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 229/429] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/projects/56fabc309a30484ebda9a05789fa5271 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:20.250130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566b46f-1319-47a9-bbe9-72f3af1e62de None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.362962 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a55d2-31fd-43db-ba24-0836d3139d32 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.363559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a55d2-31fd-43db-ba24-0836d3139d32 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.364022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a55d2-31fd-43db-ba24-0836d3139d32 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.366150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 230/430] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:20.372848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb3a53-5ba7-49f5-ad81-f20142dd9f6b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.373389 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb3a53-5ba7-49f5-ad81-f20142dd9f6b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.373864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb3a53-5ba7-49f5-ad81-f20142dd9f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.378351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566b46f-1319-47a9-bbe9-72f3af1e62de None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.389122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.400660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4ddb3a53-5ba7-49f5-ad81-f20142dd9f6b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:20.402167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4ddb3a53-5ba7-49f5-ad81-f20142dd9f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:20.419663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a6225-68dd-42b5-b6bf-52c1d055615d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.420554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a6225-68dd-42b5-b6bf-52c1d055615d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.420853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a6225-68dd-42b5-b6bf-52c1d055615d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.421108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a6225-68dd-42b5-b6bf-52c1d055615d None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.422412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a6225-68dd-42b5-b6bf-52c1d055615d None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:20.423994 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a6225-68dd-42b5-b6bf-52c1d055615d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:20.426507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 201/431] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:20.435590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a6225-68dd-42b5-b6bf-52c1d055615d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.479794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 231/432] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:20.508018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb3a53-5ba7-49f5-ad81-f20142dd9f6b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.547790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a6225-68dd-42b5-b6bf-52c1d055615d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.559765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.588477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4731c-d394-4950-b299-be790f72ae52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.589132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4731c-d394-4950-b299-be790f72ae52 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.589941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4731c-d394-4950-b299-be790f72ae52 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.590371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4731c-d394-4950-b299-be790f72ae52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.591396 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4731c-d394-4950-b299-be790f72ae52 None admin] RBAC: Adding query filter params (group.id=e902269caaf94142a68bebe80b795809, role.id=5dce8d92cffa40f79dfa1c03724c47e2, scope.project.id=56fabc309a30484ebda9a05789fa5271) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:20.591654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4731c-d394-4950-b299-be790f72ae52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:20.593247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4731c-d394-4950-b299-be790f72ae52 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:20.606611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 202/433] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/role_assignments?group.id=e902269caaf94142a68bebe80b795809&scope.project.id=56fabc309a30484ebda9a05789fa5271&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:20.627400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb3a53-5ba7-49f5-ad81-f20142dd9f6b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.666335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b738080-f63f-495d-b126-8ebaf71a2737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.668182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b738080-f63f-495d-b126-8ebaf71a2737 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.668456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b6ca8-686c-4609-98b1-52de92d4a1d1 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.668744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b6ca8-686c-4609-98b1-52de92d4a1d1 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.669022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b6ca8-686c-4609-98b1-52de92d4a1d1 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.669296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b738080-f63f-495d-b126-8ebaf71a2737 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.669755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b738080-f63f-495d-b126-8ebaf71a2737 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.670489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 203/434] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:20.676027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195f392-138b-4be3-a547-c21f99bb78bc None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.676305 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195f392-138b-4be3-a547-c21f99bb78bc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.676790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195f392-138b-4be3-a547-c21f99bb78bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.689401 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b738080-f63f-495d-b126-8ebaf71a2737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:20.694669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e195f392-138b-4be3-a547-c21f99bb78bc None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:20.695927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e195f392-138b-4be3-a547-c21f99bb78bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-7b738080-f63f-495d-b126-8ebaf71a2737 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:20.701055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:20.701996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:20.702840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:20.706860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 232/435] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:20.713218 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b738080-f63f-495d-b126-8ebaf71a2737 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.751366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.794042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 204/436] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:20.817212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195f392-138b-4be3-a547-c21f99bb78bc None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.844099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b738080-f63f-495d-b126-8ebaf71a2737 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.848475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.884970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf65ce6-ed78-4538-8627-4f9d58a3f23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.887117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65ce6-ed78-4538-8627-4f9d58a3f23d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.887855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65ce6-ed78-4538-8627-4f9d58a3f23d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.888136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65ce6-ed78-4538-8627-4f9d58a3f23d None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.889295 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf65ce6-ed78-4538-8627-4f9d58a3f23d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:20.889612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf65ce6-ed78-4538-8627-4f9d58a3f23d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:20.893051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf65ce6-ed78-4538-8627-4f9d58a3f23d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:20.906218 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 233/437] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:20.911617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf65ce6-ed78-4538-8627-4f9d58a3f23d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.968984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195f392-138b-4be3-a547-c21f99bb78bc None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.007536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ffd0d-2dba-41ec-97df-a9353189a939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.008279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ffd0d-2dba-41ec-97df-a9353189a939 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.008543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ffd0d-2dba-41ec-97df-a9353189a939 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.008813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ffd0d-2dba-41ec-97df-a9353189a939 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.025524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ffd0d-2dba-41ec-97df-a9353189a939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:21.031677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf65ce6-ed78-4538-8627-4f9d58a3f23d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-351ffd0d-2dba-41ec-97df-a9353189a939 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:21.035489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:21.039374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:21.041372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 205/438] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:21.044040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ffd0d-2dba-41ec-97df-a9353189a939 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:21.081196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15103f-3556-4c30-9fbb-8069906cfc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.083015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15103f-3556-4c30-9fbb-8069906cfc2d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.083432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15103f-3556-4c30-9fbb-8069906cfc2d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.083907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15103f-3556-4c30-9fbb-8069906cfc2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.086179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:21.123269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:21.171478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15103f-3556-4c30-9fbb-8069906cfc2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:21.172978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15103f-3556-4c30-9fbb-8069906cfc2d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:21.177764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ffd0d-2dba-41ec-97df-a9353189a939 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.215620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c751ad-2264-488b-be30-b0a7983784a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.216301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c751ad-2264-488b-be30-b0a7983784a8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.216583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c751ad-2264-488b-be30-b0a7983784a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.216858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c751ad-2264-488b-be30-b0a7983784a8 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.217608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c751ad-2264-488b-be30-b0a7983784a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:21.217918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c751ad-2264-488b-be30-b0a7983784a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:21.220196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c751ad-2264-488b-be30-b0a7983784a8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:21.232504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 206/439] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:21.233617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b8dd1-59cd-4cc0-a9cd-7b7fa84542d9 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.233900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b8dd1-59cd-4cc0-a9cd-7b7fa84542d9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.234177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b8dd1-59cd-4cc0-a9cd-7b7fa84542d9 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.235579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 207/440] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:21.239656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b8dd1-59cd-4cc0-a9cd-7b7fa84542d9 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:21.290833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 234/441] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:40:21.292328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575313-6643-42c6-9cee-c61075c9dc32 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.292652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575313-6643-42c6-9cee-c61075c9dc32 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.293226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575313-6643-42c6-9cee-c61075c9dc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.309209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-22575313-6643-42c6-9cee-c61075c9dc32 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:21.310397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-22575313-6643-42c6-9cee-c61075c9dc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:21.342443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:21.352063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b8dd1-59cd-4cc0-a9cd-7b7fa84542d9 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.377008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:21.384173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 235/442] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:21.384978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575313-6643-42c6-9cee-c61075c9dc32 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:21.387170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd902253-c5d5-4a75-b9ad-35b282823abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.387976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd902253-c5d5-4a75-b9ad-35b282823abc None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.388424 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd902253-c5d5-4a75-b9ad-35b282823abc None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.388953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd902253-c5d5-4a75-b9ad-35b282823abc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.461871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd902253-c5d5-4a75-b9ad-35b282823abc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:21.463390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd902253-c5d5-4a75-b9ad-35b282823abc None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:21.481228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575313-6643-42c6-9cee-c61075c9dc32 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.515404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c9acb-fe25-4f6b-bef9-4a682220a7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.516256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c9acb-fe25-4f6b-bef9-4a682220a7d2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.516533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c9acb-fe25-4f6b-bef9-4a682220a7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.516802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c9acb-fe25-4f6b-bef9-4a682220a7d2 None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.518017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c9acb-fe25-4f6b-bef9-4a682220a7d2 None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:21.519530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c9acb-fe25-4f6b-bef9-4a682220a7d2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:21.521204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 236/443] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:21 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:21.521943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c9acb-fe25-4f6b-bef9-4a682220a7d2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:21.554770 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:21.564915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 208/444] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:21.565814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd902253-c5d5-4a75-b9ad-35b282823abc None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:21.582805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:21.618486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c9acb-fe25-4f6b-bef9-4a682220a7d2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.651595 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb0d34-538f-4fb3-ae30-96f8f60e1dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.652405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb0d34-538f-4fb3-ae30-96f8f60e1dd9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.652688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb0d34-538f-4fb3-ae30-96f8f60e1dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.652968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb0d34-538f-4fb3-ae30-96f8f60e1dd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.662860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb0d34-538f-4fb3-ae30-96f8f60e1dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:21.664209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd902253-c5d5-4a75-b9ad-35b282823abc None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-08bb0d34-538f-4fb3-ae30-96f8f60e1dd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:21.671017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:21.671968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:21.672801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 237/445] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:21.673655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb0d34-538f-4fb3-ae30-96f8f60e1dd9 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:21.702325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043963f1-7a97-4da7-ac97-1bf6ba78842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.703218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043963f1-7a97-4da7-ac97-1bf6ba78842c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.703495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043963f1-7a97-4da7-ac97-1bf6ba78842c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.703769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043963f1-7a97-4da7-ac97-1bf6ba78842c None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.704945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043963f1-7a97-4da7-ac97-1bf6ba78842c None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:21.706372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043963f1-7a97-4da7-ac97-1bf6ba78842c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:21.708008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 209/446] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:21 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:21.708747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043963f1-7a97-4da7-ac97-1bf6ba78842c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:21.767977 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:21.772088 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb0d34-538f-4fb3-ae30-96f8f60e1dd9 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.794571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:21.804876 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2bcff-18d7-4db9-86cd-84ba1773ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.805666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2bcff-18d7-4db9-86cd-84ba1773ba3c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.805949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2bcff-18d7-4db9-86cd-84ba1773ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.806224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2bcff-18d7-4db9-86cd-84ba1773ba3c None admin] PATH_INFO: `/v3/projects/14afae0ea8a043bca3106a84957c35a0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.807435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2bcff-18d7-4db9-86cd-84ba1773ba3c None admin] RBAC: Authorizing `identity:get_project(project_id=14afae0ea8a043bca3106a84957c35a0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:21.807565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043963f1-7a97-4da7-ac97-1bf6ba78842c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.809483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2bcff-18d7-4db9-86cd-84ba1773ba3c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:21.811205 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 238/447] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:21 2022] GET /identity/v3/projects/14afae0ea8a043bca3106a84957c35a0 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:21.815308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2bcff-18d7-4db9-86cd-84ba1773ba3c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:21.840774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9814bdc-bb25-4a67-beab-4e756b8192e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.841436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814bdc-bb25-4a67-beab-4e756b8192e7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.841717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814bdc-bb25-4a67-beab-4e756b8192e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.841990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814bdc-bb25-4a67-beab-4e756b8192e7 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.842750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9814bdc-bb25-4a67-beab-4e756b8192e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:21.843058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9814bdc-bb25-4a67-beab-4e756b8192e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:21.844409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9814bdc-bb25-4a67-beab-4e756b8192e7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:21.852333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 210/448] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:21.853024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9814bdc-bb25-4a67-beab-4e756b8192e7 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:21.915637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2bcff-18d7-4db9-86cd-84ba1773ba3c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.948854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70416b90-019a-4ff6-9519-cb0459c1f329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.949503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70416b90-019a-4ff6-9519-cb0459c1f329 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.949782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70416b90-019a-4ff6-9519-cb0459c1f329 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.950053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70416b90-019a-4ff6-9519-cb0459c1f329 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.950913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70416b90-019a-4ff6-9519-cb0459c1f329 None admin] RBAC: Adding query filter params (group.id=e902269caaf94142a68bebe80b795809, role.id=73a019c7fb8340a58c7dfe4550341949, scope.project.id=14afae0ea8a043bca3106a84957c35a0) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:21.951069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9814bdc-bb25-4a67-beab-4e756b8192e7 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.951240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70416b90-019a-4ff6-9519-cb0459c1f329 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:21.952659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70416b90-019a-4ff6-9519-cb0459c1f329 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:21.958493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 239/449] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 05:40:21 2022] GET /identity/v3/role_assignments?group.id=e902269caaf94142a68bebe80b795809&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=73a019c7fb8340a58c7dfe4550341949 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:21.959104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70416b90-019a-4ff6-9519-cb0459c1f329 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:21.976216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:21.990426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2b07d-bc6b-47f5-883e-c20bc5aa94c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.991352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b07d-bc6b-47f5-883e-c20bc5aa94c5 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.991642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b07d-bc6b-47f5-883e-c20bc5aa94c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.991919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b07d-bc6b-47f5-883e-c20bc5aa94c5 None admin] PATH_INFO: `/v3/groups/e902269caaf94142a68bebe80b795809` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.993168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2b07d-bc6b-47f5-883e-c20bc5aa94c5 None admin] RBAC: Authorizing `identity:get_group(group_id=e902269caaf94142a68bebe80b795809)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:21.994686 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2b07d-bc6b-47f5-883e-c20bc5aa94c5 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:21.996985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 211/450] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:21 2022] GET /identity/v3/groups/e902269caaf94142a68bebe80b795809 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:22.003012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2b07d-bc6b-47f5-883e-c20bc5aa94c5 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:22.027720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:22.076883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70416b90-019a-4ff6-9519-cb0459c1f329 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:22.114050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2b07d-bc6b-47f5-883e-c20bc5aa94c5 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:22.115018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026df9b-92c0-4747-817a-ab8518fc4633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:22.115786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026df9b-92c0-4747-817a-ab8518fc4633 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:22.116070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026df9b-92c0-4747-817a-ab8518fc4633 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:22.116348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026df9b-92c0-4747-817a-ab8518fc4633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:22.159519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fab314-0405-47dc-99aa-83328a0f2ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:22.160187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fab314-0405-47dc-99aa-83328a0f2ed0 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:22.160890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fab314-0405-47dc-99aa-83328a0f2ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:22.161175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fab314-0405-47dc-99aa-83328a0f2ed0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:22.162367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fab314-0405-47dc-99aa-83328a0f2ed0 None admin] RBAC: Adding query filter params (group.id=e902269caaf94142a68bebe80b795809, role.id=5dce8d92cffa40f79dfa1c03724c47e2, scope.project.id=14afae0ea8a043bca3106a84957c35a0) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:22.162734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fab314-0405-47dc-99aa-83328a0f2ed0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:22.164169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fab314-0405-47dc-99aa-83328a0f2ed0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:22.172238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 212/451] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 05:40:22 2022] GET /identity/v3/role_assignments?group.id=e902269caaf94142a68bebe80b795809&scope.project.id=14afae0ea8a043bca3106a84957c35a0&role.id=5dce8d92cffa40f79dfa1c03724c47e2 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:22.191216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026df9b-92c0-4747-817a-ab8518fc4633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:22.192716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026df9b-92c0-4747-817a-ab8518fc4633 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:22.260669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:22.306998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 240/452] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:22.312693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fab314-0405-47dc-99aa-83328a0f2ed0 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:22.370145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:22.415829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fab314-0405-47dc-99aa-83328a0f2ed0 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:22.448425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195d8de-6e36-4739-8f65-a22cbf080d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:22.449857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195d8de-6e36-4739-8f65-a22cbf080d3d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:22.450141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195d8de-6e36-4739-8f65-a22cbf080d3d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:22.450411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195d8de-6e36-4739-8f65-a22cbf080d3d None admin] PATH_INFO: `/v3/projects/799d47101892496a8efa7ee2b0f5f24a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:22.451621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195d8de-6e36-4739-8f65-a22cbf080d3d None admin] RBAC: Authorizing `identity:get_project(project_id=799d47101892496a8efa7ee2b0f5f24a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:22.453057 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195d8de-6e36-4739-8f65-a22cbf080d3d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:22.454808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 213/453] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:22 2022] GET /identity/v3/projects/799d47101892496a8efa7ee2b0f5f24a => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:22.460573 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026df9b-92c0-4747-817a-ab8518fc4633 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:22.565074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026df9b-92c0-4747-817a-ab8518fc4633 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:22.599242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d7a29-36da-4914-8618-e3f49e9259e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:22.600105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d7a29-36da-4914-8618-e3f49e9259e2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:22.600390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d7a29-36da-4914-8618-e3f49e9259e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:22.600668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d7a29-36da-4914-8618-e3f49e9259e2 None admin] PATH_INFO: `/v3/groups/3f78e93462734440a78315b00ae273be` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:22.601853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d7a29-36da-4914-8618-e3f49e9259e2 None admin] RBAC: Authorizing `identity:get_group(group_id=3f78e93462734440a78315b00ae273be)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:22.603420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d7a29-36da-4914-8618-e3f49e9259e2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:22.605045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 241/454] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Jul 25 05:40:22 2022] GET /identity/v3/groups/3f78e93462734440a78315b00ae273be => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:22.610953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195d8de-6e36-4739-8f65-a22cbf080d3d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:22.711585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195d8de-6e36-4739-8f65-a22cbf080d3d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:22.733060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:22.743900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24742838-ffd5-4d2b-b572-fe3f077191ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:22.744544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24742838-ffd5-4d2b-b572-fe3f077191ac None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:22.744824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24742838-ffd5-4d2b-b572-fe3f077191ac None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:22.745099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24742838-ffd5-4d2b-b572-fe3f077191ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:22.746520 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24742838-ffd5-4d2b-b572-fe3f077191ac None admin] RBAC: Adding query filter params (group.id=3f78e93462734440a78315b00ae273be, role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=799d47101892496a8efa7ee2b0f5f24a) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:22.746832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24742838-ffd5-4d2b-b572-fe3f077191ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:22.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24742838-ffd5-4d2b-b572-fe3f077191ac None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:22.753923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 214/455] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Jul 25 05:40:22 2022] GET /identity/v3/role_assignments?group.id=3f78e93462734440a78315b00ae273be&scope.project.id=799d47101892496a8efa7ee2b0f5f24a&role.id=7a833c45603a4830b888e58e1232712a => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:25.126645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f013b30-dc0b-4e3f-9003-c3c313824b76 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.127262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f013b30-dc0b-4e3f-9003-c3c313824b76 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.127262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f013b30-dc0b-4e3f-9003-c3c313824b76 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.128806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 242/456] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:25.134138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c3a44-5860-451e-a815-ca6bf7c9ba1b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.134635 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c3a44-5860-451e-a815-ca6bf7c9ba1b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.134900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c3a44-5860-451e-a815-ca6bf7c9ba1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.162648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-924c3a44-5860-451e-a815-ca6bf7c9ba1b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:25.163775 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-924c3a44-5860-451e-a815-ca6bf7c9ba1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:25.260172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 215/457] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:25.300125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f013b30-dc0b-4e3f-9003-c3c313824b76 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:25.326084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:25.373639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b0d4f-6ba7-4174-b54c-d87b26f136da None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.373639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b0d4f-6ba7-4174-b54c-d87b26f136da None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.373639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b0d4f-6ba7-4174-b54c-d87b26f136da None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.373639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 216/458] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:25.379093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc21f0-e5d1-4501-adad-bca9dc4351d2 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.379375 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc21f0-e5d1-4501-adad-bca9dc4351d2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.379646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc21f0-e5d1-4501-adad-bca9dc4351d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.416445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-93dc21f0-e5d1-4501-adad-bca9dc4351d2 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:25.417583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-93dc21f0-e5d1-4501-adad-bca9dc4351d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:25.446867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f013b30-dc0b-4e3f-9003-c3c313824b76 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:25.507631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021f28d-74bb-4b86-8f97-22b615e00b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:25.508784 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f28d-74bb-4b86-8f97-22b615e00b15 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.509267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f28d-74bb-4b86-8f97-22b615e00b15 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.509740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f28d-74bb-4b86-8f97-22b615e00b15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.530649 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021f28d-74bb-4b86-8f97-22b615e00b15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:25.532478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021f28d-74bb-4b86-8f97-22b615e00b15 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:25.549054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 217/459] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-c021f28d-74bb-4b86-8f97-22b615e00b15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:25.560413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:25.561974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:25.562874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:25.562874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:25.562874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:25.562874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:25.564622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 243/460] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:25.570015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc21f0-e5d1-4501-adad-bca9dc4351d2 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:25.581959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021f28d-74bb-4b86-8f97-22b615e00b15 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:25.628189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:25.678576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:25.692312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc21f0-e5d1-4501-adad-bca9dc4351d2 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:25.699650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021f28d-74bb-4b86-8f97-22b615e00b15 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:25.738149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d693e7-2bbb-459b-b0ff-1bb24bad953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:25.739171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d693e7-2bbb-459b-b0ff-1bb24bad953a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.739517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d693e7-2bbb-459b-b0ff-1bb24bad953a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.740036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d693e7-2bbb-459b-b0ff-1bb24bad953a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.744451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bea1d-94e6-4609-9443-cbf8ebf280f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:25.745193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bea1d-94e6-4609-9443-cbf8ebf280f4 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.745639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bea1d-94e6-4609-9443-cbf8ebf280f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.745986 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bea1d-94e6-4609-9443-cbf8ebf280f4 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.746810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bea1d-94e6-4609-9443-cbf8ebf280f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:25.747184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bea1d-94e6-4609-9443-cbf8ebf280f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:25.749092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bea1d-94e6-4609-9443-cbf8ebf280f4 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:25.751967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d693e7-2bbb-459b-b0ff-1bb24bad953a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:25.753588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d693e7-2bbb-459b-b0ff-1bb24bad953a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:25.764762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 218/461] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:25.766150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35ae86-5e3d-48c5-b40d-934d9d0aca02 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.766435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35ae86-5e3d-48c5-b40d-934d9d0aca02 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.766732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35ae86-5e3d-48c5-b40d-934d9d0aca02 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.768215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 219/462] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:25.776262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931aea1-c462-451e-a93b-5045b0cfbf0e None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.776626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931aea1-c462-451e-a93b-5045b0cfbf0e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-92d693e7-2bbb-459b-b0ff-1bb24bad953a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:25.776763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:25.778266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:25.779368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:25.779368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:25.779368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:25.779368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:25.779643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931aea1-c462-451e-a93b-5045b0cfbf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.780685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 244/463] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:25.780996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d693e7-2bbb-459b-b0ff-1bb24bad953a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:25.801976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c931aea1-c462-451e-a93b-5045b0cfbf0e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:25.803331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c931aea1-c462-451e-a93b-5045b0cfbf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:25.882711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 220/464] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:25.883895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931aea1-c462-451e-a93b-5045b0cfbf0e None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:25.896987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d693e7-2bbb-459b-b0ff-1bb24bad953a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:25.933843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5aeaf-b760-4c0e-b5a7-0c7758c979db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:25.934963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:25.935347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5aeaf-b760-4c0e-b5a7-0c7758c979db None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.935706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5aeaf-b760-4c0e-b5a7-0c7758c979db None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.936147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5aeaf-b760-4c0e-b5a7-0c7758c979db None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.936921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5aeaf-b760-4c0e-b5a7-0c7758c979db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:25.938433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5aeaf-b760-4c0e-b5a7-0c7758c979db None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:25.942844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:25.999631 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931aea1-c462-451e-a93b-5045b0cfbf0e None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.033144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14022f3-dc98-426f-84f2-ccd5e6bbf27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.033812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14022f3-dc98-426f-84f2-ccd5e6bbf27d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.034089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14022f3-dc98-426f-84f2-ccd5e6bbf27d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.034364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14022f3-dc98-426f-84f2-ccd5e6bbf27d None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.035105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14022f3-dc98-426f-84f2-ccd5e6bbf27d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:26.035431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14022f3-dc98-426f-84f2-ccd5e6bbf27d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:26.036749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14022f3-dc98-426f-84f2-ccd5e6bbf27d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:26.043393 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 245/465] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:25 2022] POST /identity/v3/users => generated 246 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:26.043393 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5aeaf-b760-4c0e-b5a7-0c7758c979db None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.053370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 221/466] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:26.055492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bc885-7653-4eb3-bf85-61d99603e9f0 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.055843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bc885-7653-4eb3-bf85-61d99603e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.056136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bc885-7653-4eb3-bf85-61d99603e9f0 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.062657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 222/467] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:26.063853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bc885-7653-4eb3-bf85-61d99603e9f0 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.182838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:26.191290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bc885-7653-4eb3-bf85-61d99603e9f0 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.200005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5aeaf-b760-4c0e-b5a7-0c7758c979db None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.238158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a658cf-ab5a-49fb-a1fd-74ed89388b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.239097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a658cf-ab5a-49fb-a1fd-74ed89388b6a None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.239299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a658cf-ab5a-49fb-a1fd-74ed89388b6a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.239543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a658cf-ab5a-49fb-a1fd-74ed89388b6a None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.240262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a658cf-ab5a-49fb-a1fd-74ed89388b6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:26.240370 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd030825-c4ef-4a88-92c4-5b4151b468e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.241169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd030825-c4ef-4a88-92c4-5b4151b468e6 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.241452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd030825-c4ef-4a88-92c4-5b4151b468e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.241726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd030825-c4ef-4a88-92c4-5b4151b468e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.242737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a658cf-ab5a-49fb-a1fd-74ed89388b6a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:26.257375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd030825-c4ef-4a88-92c4-5b4151b468e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:26.258867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd030825-c4ef-4a88-92c4-5b4151b468e6 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-bd030825-c4ef-4a88-92c4-5b4151b468e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:26.278407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:26.279363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:26.280241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:26.280241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:26.280241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:26.280241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:26.280241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 246/468] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:26.280955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945c65-5729-4962-9cc9-d94788d85c54 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.281132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945c65-5729-4962-9cc9-d94788d85c54 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.281561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945c65-5729-4962-9cc9-d94788d85c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.298979 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9e945c65-5729-4962-9cc9-d94788d85c54 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:26.300168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9e945c65-5729-4962-9cc9-d94788d85c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:26.328543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 223/469] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:26 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:26.329478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a658cf-ab5a-49fb-a1fd-74ed89388b6a None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.338237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:26.392750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 247/470] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:26.422839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945c65-5729-4962-9cc9-d94788d85c54 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.545354 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945c65-5729-4962-9cc9-d94788d85c54 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.584712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652718ad-040d-4d9f-89a1-33ac847727af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.585571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652718ad-040d-4d9f-89a1-33ac847727af None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.585849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652718ad-040d-4d9f-89a1-33ac847727af None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.586115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652718ad-040d-4d9f-89a1-33ac847727af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.599555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652718ad-040d-4d9f-89a1-33ac847727af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:26.601194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652718ad-040d-4d9f-89a1-33ac847727af None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:26.608742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a658cf-ab5a-49fb-a1fd-74ed89388b6a None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-652718ad-040d-4d9f-89a1-33ac847727af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:26.624620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:26.625545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:26.626406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:26.626406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:26.626406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:26.626406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:26.626799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 248/471] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:26.633757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652718ad-040d-4d9f-89a1-33ac847727af None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.658554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7140e3-55b2-4e73-b267-912348401830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.659067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7140e3-55b2-4e73-b267-912348401830 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.659350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7140e3-55b2-4e73-b267-912348401830 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.659630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7140e3-55b2-4e73-b267-912348401830 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.660771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7140e3-55b2-4e73-b267-912348401830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:26.661076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7140e3-55b2-4e73-b267-912348401830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:26.663484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7140e3-55b2-4e73-b267-912348401830 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:26.670770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:26.678892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 224/472] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:26.684352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7140e3-55b2-4e73-b267-912348401830 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.716422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:26.784445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652718ad-040d-4d9f-89a1-33ac847727af None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.806794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7140e3-55b2-4e73-b267-912348401830 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.821662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1567d25-70c3-4d05-8974-0facbbd48acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.822381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567d25-70c3-4d05-8974-0facbbd48acb None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.822753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567d25-70c3-4d05-8974-0facbbd48acb None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.823114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1567d25-70c3-4d05-8974-0facbbd48acb None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.823959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1567d25-70c3-4d05-8974-0facbbd48acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:26.824326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1567d25-70c3-4d05-8974-0facbbd48acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:26.825788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1567d25-70c3-4d05-8974-0facbbd48acb None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:26.840355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 249/473] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:26.842101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5d74b-5c52-4e67-b7b0-ebb6228a5810 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.842397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5d74b-5c52-4e67-b7b0-ebb6228a5810 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.842778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5d74b-5c52-4e67-b7b0-ebb6228a5810 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.848076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a65fa-db64-4f86-a696-d209be25cd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.848225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 250/474] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:26.849077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a65fa-db64-4f86-a696-d209be25cd8e None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.849186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5d74b-5c52-4e67-b7b0-ebb6228a5810 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.849801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a65fa-db64-4f86-a696-d209be25cd8e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.850256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a65fa-db64-4f86-a696-d209be25cd8e None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.851211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a65fa-db64-4f86-a696-d209be25cd8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:26.852969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a65fa-db64-4f86-a696-d209be25cd8e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:26.938105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:26.963852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5d74b-5c52-4e67-b7b0-ebb6228a5810 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.974613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 225/475] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:26 2022] POST /identity/v3/users => generated 248 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:40:26.976188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4792fe7-a00e-4de8-be7b-e0025c538fe5 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.976475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4792fe7-a00e-4de8-be7b-e0025c538fe5 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.976804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4792fe7-a00e-4de8-be7b-e0025c538fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.014519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e4792fe7-a00e-4de8-be7b-e0025c538fe5 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:27.015787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e4792fe7-a00e-4de8-be7b-e0025c538fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:27.016294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c8e9e-ec25-483c-a3f3-a91d116de9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.017215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8e9e-ec25-483c-a3f3-a91d116de9c0 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.017706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8e9e-ec25-483c-a3f3-a91d116de9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.018210 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8e9e-ec25-483c-a3f3-a91d116de9c0 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.019178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c8e9e-ec25-483c-a3f3-a91d116de9c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.020872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c8e9e-ec25-483c-a3f3-a91d116de9c0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.108676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 251/476] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:26 2022] POST /identity/v3/users => generated 249 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:27.115893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.123015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 226/477] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:27.148765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c8e9e-ec25-483c-a3f3-a91d116de9c0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.272202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c8e9e-ec25-483c-a3f3-a91d116de9c0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.309945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744bb5f-5365-45bf-9e5b-26f3cb8efcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.310657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744bb5f-5365-45bf-9e5b-26f3cb8efcec None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.311004 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744bb5f-5365-45bf-9e5b-26f3cb8efcec None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.311285 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744bb5f-5365-45bf-9e5b-26f3cb8efcec None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.312091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744bb5f-5365-45bf-9e5b-26f3cb8efcec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.313557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744bb5f-5365-45bf-9e5b-26f3cb8efcec None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.315854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.348427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.sql.core [None req-d744bb5f-5365-45bf-9e5b-26f3cb8efcec None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 05:40:27.348427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 05:40:27.348427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [parameters: {'id': '6ffb6bcfca9848ad9acbb1e67059d6a5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 05:40:27.348427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91981) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-d744bb5f-5365-45bf-9e5b-26f3cb8efcec None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application result.read() Jul 25 05:40:27.366069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return ref.to_dict() Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:40:27.367119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application yield resource Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise self.value Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application session.commit() Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 05:40:27.368061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self.session.flush() Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application compat.raise_( Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 05:40:27.368941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:40:27.370017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application result.read() Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:40:27.372154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [parameters: {'id': '6ffb6bcfca9848ad9acbb1e67059d6a5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:27.373707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 05:40:27.375835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:27.377752 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 252/478] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Jul 25 05:40:27 2022] POST /identity/v3/roles => generated 144 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:27.377902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4792fe7-a00e-4de8-be7b-e0025c538fe5 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.484101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4792fe7-a00e-4de8-be7b-e0025c538fe5 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.531840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7df0e2-842f-4a8d-a030-6cb82f38040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.532660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7df0e2-842f-4a8d-a030-6cb82f38040e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.532937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7df0e2-842f-4a8d-a030-6cb82f38040e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.533211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7df0e2-842f-4a8d-a030-6cb82f38040e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.554205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7df0e2-842f-4a8d-a030-6cb82f38040e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-7a7df0e2-842f-4a8d-a030-6cb82f38040e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:27.562397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:27.563334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:27.564145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 227/479] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:27.569576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744bb5f-5365-45bf-9e5b-26f3cb8efcec None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.746540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744bb5f-5365-45bf-9e5b-26f3cb8efcec None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.789645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48a865-e177-4e0b-9556-f0e5b85dc444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.790708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48a865-e177-4e0b-9556-f0e5b85dc444 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.791199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48a865-e177-4e0b-9556-f0e5b85dc444 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.791717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48a865-e177-4e0b-9556-f0e5b85dc444 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.792708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48a865-e177-4e0b-9556-f0e5b85dc444 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:27.793217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48a865-e177-4e0b-9556-f0e5b85dc444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.794845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48a865-e177-4e0b-9556-f0e5b85dc444 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.810269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 253/480] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:28.431468 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89ff73-4ef9-40da-9c31-dd40dc7257bd None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.431990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89ff73-4ef9-40da-9c31-dd40dc7257bd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.432093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89ff73-4ef9-40da-9c31-dd40dc7257bd None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.433727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 228/481] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:28.439460 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a6e8b-6e4e-462f-b22c-434e457251df None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.439950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a6e8b-6e4e-462f-b22c-434e457251df None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.440047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a6e8b-6e4e-462f-b22c-434e457251df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.463650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f23a6e8b-6e4e-462f-b22c-434e457251df None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:28.465456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f23a6e8b-6e4e-462f-b22c-434e457251df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:28.541220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:28.558542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 254/482] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:28.597006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89ff73-4ef9-40da-9c31-dd40dc7257bd None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:28.676990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:28.739221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89ff73-4ef9-40da-9c31-dd40dc7257bd None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:28.782151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a6aa2-6d4b-49ff-96db-b2246bf56a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:28.783034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a6aa2-6d4b-49ff-96db-b2246bf56a53 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.783321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a6aa2-6d4b-49ff-96db-b2246bf56a53 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.783600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a6aa2-6d4b-49ff-96db-b2246bf56a53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.801325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a6aa2-6d4b-49ff-96db-b2246bf56a53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-c97a6aa2-6d4b-49ff-96db-b2246bf56a53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:28.809869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:28.810804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:28.812041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 229/483] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:28.817387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a6e8b-6e4e-462f-b22c-434e457251df None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:28.981621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a6e8b-6e4e-462f-b22c-434e457251df None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.036156 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afee070-4c39-4df5-9389-ac0b574a7002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.036766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afee070-4c39-4df5-9389-ac0b574a7002 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.037049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afee070-4c39-4df5-9389-ac0b574a7002 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.037327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afee070-4c39-4df5-9389-ac0b574a7002 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.038079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afee070-4c39-4df5-9389-ac0b574a7002 None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:29.038386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afee070-4c39-4df5-9389-ac0b574a7002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.039799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afee070-4c39-4df5-9389-ac0b574a7002 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.052851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 255/484] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:29.058675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a6aa2-6d4b-49ff-96db-b2246bf56a53 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.061968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e159f-7035-4345-90ff-848fb98c6069 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.062395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e159f-7035-4345-90ff-848fb98c6069 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.062704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e159f-7035-4345-90ff-848fb98c6069 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.064404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 256/485] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:29.070168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb8243-5469-4024-a441-93df1dfb7240 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.070457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb8243-5469-4024-a441-93df1dfb7240 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.070763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb8243-5469-4024-a441-93df1dfb7240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.095862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-94fb8243-5469-4024-a441-93df1dfb7240 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:29.097063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-94fb8243-5469-4024-a441-93df1dfb7240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:29.180915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.183807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a6aa2-6d4b-49ff-96db-b2246bf56a53 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.213437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.218738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 257/486] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 25 05:40:29.233310 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70970197-f368-4b6b-b7e7-b167b7f1b820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.234111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70970197-f368-4b6b-b7e7-b167b7f1b820 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.234387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70970197-f368-4b6b-b7e7-b167b7f1b820 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.234692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70970197-f368-4b6b-b7e7-b167b7f1b820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.246349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70970197-f368-4b6b-b7e7-b167b7f1b820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.247999 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70970197-f368-4b6b-b7e7-b167b7f1b820 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.262037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb8243-5469-4024-a441-93df1dfb7240 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-70970197-f368-4b6b-b7e7-b167b7f1b820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:29.283184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:29.284108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:29.284982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:29.284982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:29.284982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:29.284982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:29.284982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 230/487] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:29.295321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70970197-f368-4b6b-b7e7-b167b7f1b820 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.409765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70970197-f368-4b6b-b7e7-b167b7f1b820 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.445019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a74b7d-5f66-47bd-95c7-78c01d26c18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.445738 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a74b7d-5f66-47bd-95c7-78c01d26c18b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.446174 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a74b7d-5f66-47bd-95c7-78c01d26c18b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.446549 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a74b7d-5f66-47bd-95c7-78c01d26c18b None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.447352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a74b7d-5f66-47bd-95c7-78c01d26c18b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:29.447732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a74b7d-5f66-47bd-95c7-78c01d26c18b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.449422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a74b7d-5f66-47bd-95c7-78c01d26c18b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.462211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 231/488] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:29.467566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a74b7d-5f66-47bd-95c7-78c01d26c18b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.469357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb8243-5469-4024-a441-93df1dfb7240 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.521607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5127b-4755-4636-9a5b-f5e55f3f1334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.522419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5127b-4755-4636-9a5b-f5e55f3f1334 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.522731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5127b-4755-4636-9a5b-f5e55f3f1334 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.523008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5127b-4755-4636-9a5b-f5e55f3f1334 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.529490 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.535966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5127b-4755-4636-9a5b-f5e55f3f1334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-c9f5127b-4755-4636-9a5b-f5e55f3f1334 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:29.548121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:29.549099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:29.549919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:29.554502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 258/489] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:29.554502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f808fc-9eaf-4980-a084-f1631469230c None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.554502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f808fc-9eaf-4980-a084-f1631469230c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.554502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f808fc-9eaf-4980-a084-f1631469230c None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.555711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 259/490] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:29.556862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f808fc-9eaf-4980-a084-f1631469230c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.603896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.622854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a74b7d-5f66-47bd-95c7-78c01d26c18b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.660518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a434a6-f0f6-4c04-b9ec-251ac3bcab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.661867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a434a6-f0f6-4c04-b9ec-251ac3bcab2b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.662205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a434a6-f0f6-4c04-b9ec-251ac3bcab2b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.662419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a434a6-f0f6-4c04-b9ec-251ac3bcab2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.671982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a434a6-f0f6-4c04-b9ec-251ac3bcab2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.673828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a434a6-f0f6-4c04-b9ec-251ac3bcab2b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.682790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f808fc-9eaf-4980-a084-f1631469230c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-b4a434a6-f0f6-4c04-b9ec-251ac3bcab2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:29.691200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:29.692126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:29.693089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 232/491] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:29.694168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6dd87-29aa-4443-972e-60e150e3efe1 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.694340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6dd87-29aa-4443-972e-60e150e3efe1 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.698534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6dd87-29aa-4443-972e-60e150e3efe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.714976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-b4d6dd87-29aa-4443-972e-60e150e3efe1 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:29.716339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b4d6dd87-29aa-4443-972e-60e150e3efe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:29.720322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d5626-bd4f-4716-abce-a7e5ffad901e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.720998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d5626-bd4f-4716-abce-a7e5ffad901e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.721275 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d5626-bd4f-4716-abce-a7e5ffad901e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.721546 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d5626-bd4f-4716-abce-a7e5ffad901e None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.722286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d5626-bd4f-4716-abce-a7e5ffad901e None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:29.722628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d5626-bd4f-4716-abce-a7e5ffad901e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.724516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d5626-bd4f-4716-abce-a7e5ffad901e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.738543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 260/492] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:29.740112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c29d22-9ad2-4f12-b1b3-3ce70d0ba337 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.740235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c29d22-9ad2-4f12-b1b3-3ce70d0ba337 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.740486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c29d22-9ad2-4f12-b1b3-3ce70d0ba337 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.741888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 261/493] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:29.745820 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c29d22-9ad2-4f12-b1b3-3ce70d0ba337 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.794984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 233/494] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:29.796382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6dd87-29aa-4443-972e-60e150e3efe1 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.821310 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.862756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c29d22-9ad2-4f12-b1b3-3ce70d0ba337 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.865046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.898926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0d376-95c8-4d1f-8ac8-7f46a6069883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.899594 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0d376-95c8-4d1f-8ac8-7f46a6069883 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.899919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0d376-95c8-4d1f-8ac8-7f46a6069883 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.900189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0d376-95c8-4d1f-8ac8-7f46a6069883 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.901047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0d376-95c8-4d1f-8ac8-7f46a6069883 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:29.901354 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0d376-95c8-4d1f-8ac8-7f46a6069883 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.903437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0d376-95c8-4d1f-8ac8-7f46a6069883 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.904264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6dd87-29aa-4443-972e-60e150e3efe1 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.939041 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 262/495] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:29.940304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d401b17-ed7c-4f12-966e-a4d18fc5891d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.940594 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d401b17-ed7c-4f12-966e-a4d18fc5891d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.940870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d401b17-ed7c-4f12-966e-a4d18fc5891d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.945719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742044d-a2a9-4ed0-954d-9d629fb9575e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.946540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742044d-a2a9-4ed0-954d-9d629fb9575e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.946826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742044d-a2a9-4ed0-954d-9d629fb9575e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.947101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742044d-a2a9-4ed0-954d-9d629fb9575e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.958280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742044d-a2a9-4ed0-954d-9d629fb9575e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.958977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-5d401b17-ed7c-4f12-966e-a4d18fc5891d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:29.959835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742044d-a2a9-4ed0-954d-9d629fb9575e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.960146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5d401b17-ed7c-4f12-966e-a4d18fc5891d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-f742044d-a2a9-4ed0-954d-9d629fb9575e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:29.978155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:29.979241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:29.980150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:29.980150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:29.980150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:29.980150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:29.984145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 234/496] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:29.984961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742044d-a2a9-4ed0-954d-9d629fb9575e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.029283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 263/497] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:30.030126 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d401b17-ed7c-4f12-966e-a4d18fc5891d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.078511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:30.110878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742044d-a2a9-4ed0-954d-9d629fb9575e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:30.131595 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:30.141269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d401b17-ed7c-4f12-966e-a4d18fc5891d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:30.149891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29019c93-93df-4b36-82cb-841466e9db3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:30.150952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29019c93-93df-4b36-82cb-841466e9db3f None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.151415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29019c93-93df-4b36-82cb-841466e9db3f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.152459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29019c93-93df-4b36-82cb-841466e9db3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.163381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29019c93-93df-4b36-82cb-841466e9db3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-29019c93-93df-4b36-82cb-841466e9db3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:30.171218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:30.172111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:30.173007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 235/498] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:30.173684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29019c93-93df-4b36-82cb-841466e9db3f None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.180874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a94a6-221f-417c-9947-973fc545e546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:30.181051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a94a6-221f-417c-9947-973fc545e546 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.181290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a94a6-221f-417c-9947-973fc545e546 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.181571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a94a6-221f-417c-9947-973fc545e546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.253005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a94a6-221f-417c-9947-973fc545e546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:30.255066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a94a6-221f-417c-9947-973fc545e546 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:30.291534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29019c93-93df-4b36-82cb-841466e9db3f None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:30.326102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:30.332124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22d321-5e3f-4529-9134-b0106c7372c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:30.332734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22d321-5e3f-4529-9134-b0106c7372c4 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.333017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22d321-5e3f-4529-9134-b0106c7372c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.333300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22d321-5e3f-4529-9134-b0106c7372c4 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.334041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22d321-5e3f-4529-9134-b0106c7372c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:30.334348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22d321-5e3f-4529-9134-b0106c7372c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:30.336160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22d321-5e3f-4529-9134-b0106c7372c4 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:30.348616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 236/499] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:30.349452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22d321-5e3f-4529-9134-b0106c7372c4 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.357229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:30.366884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 264/500] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:30.372444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a94a6-221f-417c-9947-973fc545e546 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.459517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22d321-5e3f-4529-9134-b0106c7372c4 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:30.487060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a94a6-221f-417c-9947-973fc545e546 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:30.494682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50389739-dcb1-4de8-b6f5-ba1cfcc0784b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:30.496015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50389739-dcb1-4de8-b6f5-ba1cfcc0784b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.496296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50389739-dcb1-4de8-b6f5-ba1cfcc0784b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.496567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50389739-dcb1-4de8-b6f5-ba1cfcc0784b None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.506835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50389739-dcb1-4de8-b6f5-ba1cfcc0784b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-50389739-dcb1-4de8-b6f5-ba1cfcc0784b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:30.515479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:30.516598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:30.517493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 237/501] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:30.518174 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50389739-dcb1-4de8-b6f5-ba1cfcc0784b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.529812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb92b7-697f-431e-b1bf-ae0ba1aebd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:30.530500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb92b7-697f-431e-b1bf-ae0ba1aebd03 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.530788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb92b7-697f-431e-b1bf-ae0ba1aebd03 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.531063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb92b7-697f-431e-b1bf-ae0ba1aebd03 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.531815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb92b7-697f-431e-b1bf-ae0ba1aebd03 None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:30.532121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb92b7-697f-431e-b1bf-ae0ba1aebd03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:30.533490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb92b7-697f-431e-b1bf-ae0ba1aebd03 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:30.541826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 265/502] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:30.542676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb92b7-697f-431e-b1bf-ae0ba1aebd03 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.581825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:30.609091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:30.630546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50389739-dcb1-4de8-b6f5-ba1cfcc0784b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:30.652334 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb92b7-697f-431e-b1bf-ae0ba1aebd03 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:30.663801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc103e0-1ed9-4cd9-af05-360ced040468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:30.664606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc103e0-1ed9-4cd9-af05-360ced040468 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.664891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc103e0-1ed9-4cd9-af05-360ced040468 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.665180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc103e0-1ed9-4cd9-af05-360ced040468 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.673116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc103e0-1ed9-4cd9-af05-360ced040468 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:30.674590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc103e0-1ed9-4cd9-af05-360ced040468 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:30.686058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fddbd-4b04-4393-b2ca-12a0c2fdb0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:30.686886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fddbd-4b04-4393-b2ca-12a0c2fdb0eb None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.687167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fddbd-4b04-4393-b2ca-12a0c2fdb0eb None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.687441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fddbd-4b04-4393-b2ca-12a0c2fdb0eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-afc103e0-1ed9-4cd9-af05-360ced040468 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:30.689199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:30.690255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 238/503] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:30.696206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc103e0-1ed9-4cd9-af05-360ced040468 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.700977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fddbd-4b04-4393-b2ca-12a0c2fdb0eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:30.702444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fddbd-4b04-4393-b2ca-12a0c2fdb0eb None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-6a3fddbd-4b04-4393-b2ca-12a0c2fdb0eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:30.722173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:30.723253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:30.724727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:30.724727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:30.724727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:30.724727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:30.724727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 266/504] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:30.724727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fddbd-4b04-4393-b2ca-12a0c2fdb0eb None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.796989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc103e0-1ed9-4cd9-af05-360ced040468 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:30.809801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:30.823035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fddbd-4b04-4393-b2ca-12a0c2fdb0eb None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:30.829823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee8ef6-9df9-439b-8a17-52c64b7be097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:30.829966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:30.830516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee8ef6-9df9-439b-8a17-52c64b7be097 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.830801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee8ef6-9df9-439b-8a17-52c64b7be097 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.831078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee8ef6-9df9-439b-8a17-52c64b7be097 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.831820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee8ef6-9df9-439b-8a17-52c64b7be097 None admin] RBAC: Adding query filter params (name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:30.832127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee8ef6-9df9-439b-8a17-52c64b7be097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:30.833492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee8ef6-9df9-439b-8a17-52c64b7be097 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:30.841647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 239/505] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:30.842904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e52a37-816d-4732-9f17-739bb2494969 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.843584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e52a37-816d-4732-9f17-739bb2494969 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.844102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e52a37-816d-4732-9f17-739bb2494969 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.846131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 240/506] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:30.846990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e52a37-816d-4732-9f17-739bb2494969 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.862829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47053df-938f-4bc9-9cef-55134ab32608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:30.863550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47053df-938f-4bc9-9cef-55134ab32608 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.863833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47053df-938f-4bc9-9cef-55134ab32608 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.864107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47053df-938f-4bc9-9cef-55134ab32608 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.874759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47053df-938f-4bc9-9cef-55134ab32608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:30.876221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47053df-938f-4bc9-9cef-55134ab32608 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-b47053df-938f-4bc9-9cef-55134ab32608 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:30.893926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:30.894844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:30.895830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:30.895830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:30.895830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:30.895830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:30.895830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 267/507] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:30.896269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47053df-938f-4bc9-9cef-55134ab32608 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:30.952435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e52a37-816d-4732-9f17-739bb2494969 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:30.985504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08418e-b43c-4001-b576-eaab231f6d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:30.986166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08418e-b43c-4001-b576-eaab231f6d54 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:30.986447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08418e-b43c-4001-b576-eaab231f6d54 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:30.986780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08418e-b43c-4001-b576-eaab231f6d54 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:30.987606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08418e-b43c-4001-b576-eaab231f6d54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:30.987912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08418e-b43c-4001-b576-eaab231f6d54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:30.989342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08418e-b43c-4001-b576-eaab231f6d54 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:30.994899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47053df-938f-4bc9-9cef-55134ab32608 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.025455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 241/508] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 05:40:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:31.026264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08418e-b43c-4001-b576-eaab231f6d54 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.028302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bcef6-eaeb-47b6-8faf-04d487d06c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.028968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bcef6-eaeb-47b6-8faf-04d487d06c04 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.029258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bcef6-eaeb-47b6-8faf-04d487d06c04 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.029535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bcef6-eaeb-47b6-8faf-04d487d06c04 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.030265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bcef6-eaeb-47b6-8faf-04d487d06c04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:31.030604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bcef6-eaeb-47b6-8faf-04d487d06c04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.032008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bcef6-eaeb-47b6-8faf-04d487d06c04 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.044662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 268/509] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:31.045941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e72b9-17b2-41e0-8b74-d41a53709aa2 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.046238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e72b9-17b2-41e0-8b74-d41a53709aa2 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.046543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e72b9-17b2-41e0-8b74-d41a53709aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.063217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-0e3e72b9-17b2-41e0-8b74-d41a53709aa2 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:31.064447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0e3e72b9-17b2-41e0-8b74-d41a53709aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:31.066183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.077983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.137837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 269/510] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:31.137837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e72b9-17b2-41e0-8b74-d41a53709aa2 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.138087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08418e-b43c-4001-b576-eaab231f6d54 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.173280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4f0b9-47cc-4bdd-98f6-892a2dc2302e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.174075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f0b9-47cc-4bdd-98f6-892a2dc2302e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.174352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f0b9-47cc-4bdd-98f6-892a2dc2302e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.174653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f0b9-47cc-4bdd-98f6-892a2dc2302e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.184682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4f0b9-47cc-4bdd-98f6-892a2dc2302e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.186175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4f0b9-47cc-4bdd-98f6-892a2dc2302e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-edf4f0b9-47cc-4bdd-98f6-892a2dc2302e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:31.203193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:31.204052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:31.204922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:31.204922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:31.204922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:31.204922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:31.204922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 242/511] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:31.205187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4f0b9-47cc-4bdd-98f6-892a2dc2302e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.244946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e72b9-17b2-41e0-8b74-d41a53709aa2 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.278020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecf7b3-5af8-40d7-aa9a-52ff25e2bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.278705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecf7b3-5af8-40d7-aa9a-52ff25e2bc58 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.278987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecf7b3-5af8-40d7-aa9a-52ff25e2bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.279260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecf7b3-5af8-40d7-aa9a-52ff25e2bc58 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.279981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecf7b3-5af8-40d7-aa9a-52ff25e2bc58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:31.280286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecf7b3-5af8-40d7-aa9a-52ff25e2bc58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.281668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecf7b3-5af8-40d7-aa9a-52ff25e2bc58 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.290067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.293468 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 270/512] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:31.294264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecf7b3-5af8-40d7-aa9a-52ff25e2bc58 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.304963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.306225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4f0b9-47cc-4bdd-98f6-892a2dc2302e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.340793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b2857-c24e-4b75-a84c-bc4f50871cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.341442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b2857-c24e-4b75-a84c-bc4f50871cd6 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.341723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b2857-c24e-4b75-a84c-bc4f50871cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.342000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b2857-c24e-4b75-a84c-bc4f50871cd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.396409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecf7b3-5af8-40d7-aa9a-52ff25e2bc58 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.413749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b2857-c24e-4b75-a84c-bc4f50871cd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.415260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b2857-c24e-4b75-a84c-bc4f50871cd6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.435937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce1c9e-e58a-4da4-b383-2e52f150ad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.436739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce1c9e-e58a-4da4-b383-2e52f150ad99 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.437020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce1c9e-e58a-4da4-b383-2e52f150ad99 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.437293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce1c9e-e58a-4da4-b383-2e52f150ad99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.444078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce1c9e-e58a-4da4-b383-2e52f150ad99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.445523 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce1c9e-e58a-4da4-b383-2e52f150ad99 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-79ce1c9e-e58a-4da4-b383-2e52f150ad99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:31.460141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:31.478097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:31.479027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:31.479027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:31.479027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:31.479027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:31.479027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:31.479027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:31.479027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:31.479027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:31.479027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 271/513] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:31.479027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce1c9e-e58a-4da4-b383-2e52f150ad99 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.506008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.521961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 243/514] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:31.522949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b2857-c24e-4b75-a84c-bc4f50871cd6 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.588598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.621756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce1c9e-e58a-4da4-b383-2e52f150ad99 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.650813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b2857-c24e-4b75-a84c-bc4f50871cd6 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.654589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249a98b-2b36-49c5-89a8-951a8dabca71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.655371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a98b-2b36-49c5-89a8-951a8dabca71 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.655651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a98b-2b36-49c5-89a8-951a8dabca71 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.655926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a98b-2b36-49c5-89a8-951a8dabca71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.665912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249a98b-2b36-49c5-89a8-951a8dabca71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.667928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249a98b-2b36-49c5-89a8-951a8dabca71 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.683408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee026d0-947c-478f-a301-51bf50b390e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.684066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee026d0-947c-478f-a301-51bf50b390e3 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.684342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee026d0-947c-478f-a301-51bf50b390e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.684610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee026d0-947c-478f-a301-51bf50b390e3 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-c249a98b-2b36-49c5-89a8-951a8dabca71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:31.685064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:31.685946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:31.686843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:31.686843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:31.686843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:31.686843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:31.687052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee026d0-947c-478f-a301-51bf50b390e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:31.687052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee026d0-947c-478f-a301-51bf50b390e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.687179 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 272/515] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:31.687247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee026d0-947c-478f-a301-51bf50b390e3 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.687641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249a98b-2b36-49c5-89a8-951a8dabca71 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.698836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 244/516] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:31.699753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee026d0-947c-478f-a301-51bf50b390e3 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.751688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.788801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249a98b-2b36-49c5-89a8-951a8dabca71 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.804987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee026d0-947c-478f-a301-51bf50b390e3 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.808383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.821622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e183843-7220-4c82-8c90-900fc21ce32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.822406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e183843-7220-4c82-8c90-900fc21ce32c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.826367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e183843-7220-4c82-8c90-900fc21ce32c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.826367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e183843-7220-4c82-8c90-900fc21ce32c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.830113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e183843-7220-4c82-8c90-900fc21ce32c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.831600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e183843-7220-4c82-8c90-900fc21ce32c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.837319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281ed9c-c917-40e0-b4c4-b9190d68e2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.838023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281ed9c-c917-40e0-b4c4-b9190d68e2ec None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.838302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281ed9c-c917-40e0-b4c4-b9190d68e2ec None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.838610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281ed9c-c917-40e0-b4c4-b9190d68e2ec None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.840207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281ed9c-c917-40e0-b4c4-b9190d68e2ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:31.840515 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281ed9c-c917-40e0-b4c4-b9190d68e2ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.841934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281ed9c-c917-40e0-b4c4-b9190d68e2ec None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5e183843-7220-4c82-8c90-900fc21ce32c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:31.848383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:31.849442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 273/517] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:31.850390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e183843-7220-4c82-8c90-900fc21ce32c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.864983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 245/518] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:31.864983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281ed9c-c917-40e0-b4c4-b9190d68e2ec None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.958444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e183843-7220-4c82-8c90-900fc21ce32c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.960822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281ed9c-c917-40e0-b4c4-b9190d68e2ec None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.963148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.991807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac57681-a6e9-4d3e-97d4-74010a4fa461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.992599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac57681-a6e9-4d3e-97d4-74010a4fa461 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.992881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac57681-a6e9-4d3e-97d4-74010a4fa461 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.993155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac57681-a6e9-4d3e-97d4-74010a4fa461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.993899 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676efffa-4d2f-4c47-8ddc-2ecceb18b0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.994627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676efffa-4d2f-4c47-8ddc-2ecceb18b0df None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.994957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676efffa-4d2f-4c47-8ddc-2ecceb18b0df None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.995238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676efffa-4d2f-4c47-8ddc-2ecceb18b0df None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.995976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676efffa-4d2f-4c47-8ddc-2ecceb18b0df None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:31.996284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676efffa-4d2f-4c47-8ddc-2ecceb18b0df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.997654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676efffa-4d2f-4c47-8ddc-2ecceb18b0df None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.003831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac57681-a6e9-4d3e-97d4-74010a4fa461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.005293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac57681-a6e9-4d3e-97d4-74010a4fa461 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.009406 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 246/519] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:32.010200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676efffa-4d2f-4c47-8ddc-2ecceb18b0df None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-aac57681-a6e9-4d3e-97d4-74010a4fa461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:32.022713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:32.023751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:32.029751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:32.029751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:32.029751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:32.029751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:32.029751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 274/520] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:32.029751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac57681-a6e9-4d3e-97d4-74010a4fa461 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.055722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.113154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676efffa-4d2f-4c47-8ddc-2ecceb18b0df None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.122332 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac57681-a6e9-4d3e-97d4-74010a4fa461 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.146020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee6717-fb1e-452c-9144-c96aca6f1aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.146865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee6717-fb1e-452c-9144-c96aca6f1aab None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.147144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee6717-fb1e-452c-9144-c96aca6f1aab None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.147418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee6717-fb1e-452c-9144-c96aca6f1aab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.154795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee6717-fb1e-452c-9144-c96aca6f1aab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.155837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36f25c-d7de-49e5-af71-1209856ae868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.156245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee6717-fb1e-452c-9144-c96aca6f1aab None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.156496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f25c-d7de-49e5-af71-1209856ae868 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.156782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f25c-d7de-49e5-af71-1209856ae868 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.157059 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f25c-d7de-49e5-af71-1209856ae868 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.157941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36f25c-d7de-49e5-af71-1209856ae868 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:32.158202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36f25c-d7de-49e5-af71-1209856ae868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.159652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36f25c-d7de-49e5-af71-1209856ae868 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-a2ee6717-fb1e-452c-9144-c96aca6f1aab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:32.170381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:32.171474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 247/521] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:32.172400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee6717-fb1e-452c-9144-c96aca6f1aab None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.179892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 275/522] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:32.180983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36f25c-d7de-49e5-af71-1209856ae868 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.197062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.270311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.270531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee6717-fb1e-452c-9144-c96aca6f1aab None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.280469 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36f25c-d7de-49e5-af71-1209856ae868 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.305229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8a87a-0c32-4472-a481-4f07267efde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.305887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8a87a-0c32-4472-a481-4f07267efde3 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.306167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8a87a-0c32-4472-a481-4f07267efde3 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.306443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8a87a-0c32-4472-a481-4f07267efde3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.307296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8a87a-0c32-4472-a481-4f07267efde3 None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=067b827625e0427294b93b027597a5f3) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:32.307604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8a87a-0c32-4472-a481-4f07267efde3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.309092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8a87a-0c32-4472-a481-4f07267efde3 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.313604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3a39-fdc5-47bc-bbbb-ce2f37236064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.314272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3a39-fdc5-47bc-bbbb-ce2f37236064 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.314587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3a39-fdc5-47bc-bbbb-ce2f37236064 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.314682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 248/523] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/role_assignments?user.id=067b827625e0427294b93b027597a5f3&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:32.314864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3a39-fdc5-47bc-bbbb-ce2f37236064 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.315451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8a87a-0c32-4472-a481-4f07267efde3 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.315615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd3a39-fdc5-47bc-bbbb-ce2f37236064 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.317095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd3a39-fdc5-47bc-bbbb-ce2f37236064 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.403182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 276/524] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:32 2022] POST /identity/v3/users => generated 247 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:32.404042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3a39-fdc5-47bc-bbbb-ce2f37236064 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.429065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.449639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8a87a-0c32-4472-a481-4f07267efde3 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.500187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88145477-a3d1-41d3-b95d-6401374a842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.500876 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145477-a3d1-41d3-b95d-6401374a842b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.501162 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145477-a3d1-41d3-b95d-6401374a842b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.501436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145477-a3d1-41d3-b95d-6401374a842b None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.502182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88145477-a3d1-41d3-b95d-6401374a842b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:32.502512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88145477-a3d1-41d3-b95d-6401374a842b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.504709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88145477-a3d1-41d3-b95d-6401374a842b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.521042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 249/525] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:32.521911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88145477-a3d1-41d3-b95d-6401374a842b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.537839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3a39-fdc5-47bc-bbbb-ce2f37236064 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.576366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b8c7c-2ed4-4d84-9e9f-d4e0fb1701b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.577175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b8c7c-2ed4-4d84-9e9f-d4e0fb1701b2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.577637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b8c7c-2ed4-4d84-9e9f-d4e0fb1701b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.578015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b8c7c-2ed4-4d84-9e9f-d4e0fb1701b2 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.579136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b8c7c-2ed4-4d84-9e9f-d4e0fb1701b2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:32.579546 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b8c7c-2ed4-4d84-9e9f-d4e0fb1701b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.581228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b8c7c-2ed4-4d84-9e9f-d4e0fb1701b2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.602612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 277/526] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:32.603852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b8c7c-2ed4-4d84-9e9f-d4e0fb1701b2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.609580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.655502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88145477-a3d1-41d3-b95d-6401374a842b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.701883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e4987-50ff-45d6-965f-fc3016c62da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.703468 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e4987-50ff-45d6-965f-fc3016c62da6 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.703986 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e4987-50ff-45d6-965f-fc3016c62da6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.704668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e4987-50ff-45d6-965f-fc3016c62da6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.724961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b8c7c-2ed4-4d84-9e9f-d4e0fb1701b2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.732194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.771089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b766a96-0b69-4095-be9b-06c5998aaae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.772059 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b766a96-0b69-4095-be9b-06c5998aaae1 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.772430 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b766a96-0b69-4095-be9b-06c5998aaae1 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.772804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b766a96-0b69-4095-be9b-06c5998aaae1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.780896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b766a96-0b69-4095-be9b-06c5998aaae1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.782624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b766a96-0b69-4095-be9b-06c5998aaae1 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.797563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e4987-50ff-45d6-965f-fc3016c62da6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-9b766a96-0b69-4095-be9b-06c5998aaae1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:32.798707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:32.799711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:32.800586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:32.800586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:32.800586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:32.800586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:32.800586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:32.800586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:32.800586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:32.800586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:32.801994 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e4987-50ff-45d6-965f-fc3016c62da6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.805982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 278/527] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:32.806935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b766a96-0b69-4095-be9b-06c5998aaae1 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.884310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.914857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 250/528] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:32.916090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e4987-50ff-45d6-965f-fc3016c62da6 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.932021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b766a96-0b69-4095-be9b-06c5998aaae1 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.969618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8f44e-e5e4-4797-8926-d7261b179b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.970201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8f44e-e5e4-4797-8926-d7261b179b60 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.970508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8f44e-e5e4-4797-8926-d7261b179b60 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.970795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8f44e-e5e4-4797-8926-d7261b179b60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.971882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.044282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e4987-50ff-45d6-965f-fc3016c62da6 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.054992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8f44e-e5e4-4797-8926-d7261b179b60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.056572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8f44e-e5e4-4797-8926-d7261b179b60 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.091668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9640078-ebc5-44b2-81d8-3018b8c1a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.092623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9640078-ebc5-44b2-81d8-3018b8c1a67e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.092909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9640078-ebc5-44b2-81d8-3018b8c1a67e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.093187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9640078-ebc5-44b2-81d8-3018b8c1a67e None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.093982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9640078-ebc5-44b2-81d8-3018b8c1a67e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:33.094318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9640078-ebc5-44b2-81d8-3018b8c1a67e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.096020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9640078-ebc5-44b2-81d8-3018b8c1a67e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.115744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 251/529] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:33.117428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9640078-ebc5-44b2-81d8-3018b8c1a67e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.142296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.195862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 279/530] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:33.196930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8f44e-e5e4-4797-8926-d7261b179b60 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.236893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9640078-ebc5-44b2-81d8-3018b8c1a67e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.246356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.273292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b82e6-c9b6-4a41-8d93-052c3d6348af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.274345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b82e6-c9b6-4a41-8d93-052c3d6348af None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.274670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b82e6-c9b6-4a41-8d93-052c3d6348af None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.274949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b82e6-c9b6-4a41-8d93-052c3d6348af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.291030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b82e6-c9b6-4a41-8d93-052c3d6348af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.292552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b82e6-c9b6-4a41-8d93-052c3d6348af None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-e68b82e6-c9b6-4a41-8d93-052c3d6348af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:33.313544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:33.314565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:33.315438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:33.315438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:33.315438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:33.315438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:33.321469 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 252/531] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:33.322581 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b82e6-c9b6-4a41-8d93-052c3d6348af None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.326524 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8f44e-e5e4-4797-8926-d7261b179b60 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.362427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5a83b-dfe8-478a-9b6a-450651d01a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.363112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5a83b-dfe8-478a-9b6a-450651d01a32 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.363392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5a83b-dfe8-478a-9b6a-450651d01a32 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.363661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5a83b-dfe8-478a-9b6a-450651d01a32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.364490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5a83b-dfe8-478a-9b6a-450651d01a32 None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=6e12361da5f849ee97607ca68ee6ef10) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:33.364796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5a83b-dfe8-478a-9b6a-450651d01a32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.366719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5a83b-dfe8-478a-9b6a-450651d01a32 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.378519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 280/532] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/role_assignments?user.id=6e12361da5f849ee97607ca68ee6ef10&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:33.378519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5a83b-dfe8-478a-9b6a-450651d01a32 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.416699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.505263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5a83b-dfe8-478a-9b6a-450651d01a32 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.546860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2dfa4-f344-452d-b7ee-f3ed3e534990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.547746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2dfa4-f344-452d-b7ee-f3ed3e534990 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.548035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2dfa4-f344-452d-b7ee-f3ed3e534990 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.548309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2dfa4-f344-452d-b7ee-f3ed3e534990 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.549044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2dfa4-f344-452d-b7ee-f3ed3e534990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:33.549407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2dfa4-f344-452d-b7ee-f3ed3e534990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.550978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2dfa4-f344-452d-b7ee-f3ed3e534990 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.565308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 281/533] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:33.570504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2dfa4-f344-452d-b7ee-f3ed3e534990 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.572458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b82e6-c9b6-4a41-8d93-052c3d6348af None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.681061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2dfa4-f344-452d-b7ee-f3ed3e534990 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.684977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.694239 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f325ea2-7cad-4c0b-bbc5-e3df8e95eb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.695142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f325ea2-7cad-4c0b-bbc5-e3df8e95eb30 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.695433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f325ea2-7cad-4c0b-bbc5-e3df8e95eb30 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.695713 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f325ea2-7cad-4c0b-bbc5-e3df8e95eb30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.707061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f325ea2-7cad-4c0b-bbc5-e3df8e95eb30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.708562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f325ea2-7cad-4c0b-bbc5-e3df8e95eb30 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-2f325ea2-7cad-4c0b-bbc5-e3df8e95eb30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:33.726928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:33.730251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:33.733862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:33.733862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:33.733862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:33.733862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:33.733862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 253/534] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:33.739442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f325ea2-7cad-4c0b-bbc5-e3df8e95eb30 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.749890 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8589f2-4b01-4ab8-8ebf-23a4eca87dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.750740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8589f2-4b01-4ab8-8ebf-23a4eca87dc4 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.751050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8589f2-4b01-4ab8-8ebf-23a4eca87dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.751400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8589f2-4b01-4ab8-8ebf-23a4eca87dc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.755608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.759782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8589f2-4b01-4ab8-8ebf-23a4eca87dc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.761278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8589f2-4b01-4ab8-8ebf-23a4eca87dc4 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-9b8589f2-4b01-4ab8-8ebf-23a4eca87dc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:33.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:33.794558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:33.796692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:33.796692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:33.796692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:33.796692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:33.796692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:33.796692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:33.796692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:33.796692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:33.796692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 282/535] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:33.805229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8589f2-4b01-4ab8-8ebf-23a4eca87dc4 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.843197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f325ea2-7cad-4c0b-bbc5-e3df8e95eb30 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.878519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6b798-825d-4380-bf2a-1efe0d7ae888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.879195 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6b798-825d-4380-bf2a-1efe0d7ae888 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.879483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6b798-825d-4380-bf2a-1efe0d7ae888 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.879764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6b798-825d-4380-bf2a-1efe0d7ae888 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.880485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6b798-825d-4380-bf2a-1efe0d7ae888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:33.880793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6b798-825d-4380-bf2a-1efe0d7ae888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.882512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6b798-825d-4380-bf2a-1efe0d7ae888 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.893975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 254/536] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:33.899637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6b798-825d-4380-bf2a-1efe0d7ae888 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.982023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:34.002034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6b798-825d-4380-bf2a-1efe0d7ae888 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.041342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8589f2-4b01-4ab8-8ebf-23a4eca87dc4 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.043047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932e675-e855-4fdb-bd9e-b5cbd38dee88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.043266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932e675-e855-4fdb-bd9e-b5cbd38dee88 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.043771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932e675-e855-4fdb-bd9e-b5cbd38dee88 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.044224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932e675-e855-4fdb-bd9e-b5cbd38dee88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.051893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932e675-e855-4fdb-bd9e-b5cbd38dee88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.053427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932e675-e855-4fdb-bd9e-b5cbd38dee88 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-d932e675-e855-4fdb-bd9e-b5cbd38dee88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:34.068414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:34.069489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:34.070592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:34.070592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:34.070592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:34.070592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:34.070592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:34.070592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:34.070592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:34.070592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:34.072479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 255/537] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:34.077154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932e675-e855-4fdb-bd9e-b5cbd38dee88 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.110448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ce08a-5b40-4128-91a5-e4c1cfd7080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.111141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ce08a-5b40-4128-91a5-e4c1cfd7080f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.111419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ce08a-5b40-4128-91a5-e4c1cfd7080f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.111692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ce08a-5b40-4128-91a5-e4c1cfd7080f None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.112490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ce08a-5b40-4128-91a5-e4c1cfd7080f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:34.112898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ce08a-5b40-4128-91a5-e4c1cfd7080f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.114430 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ce08a-5b40-4128-91a5-e4c1cfd7080f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.146762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 283/538] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:34.152688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ce08a-5b40-4128-91a5-e4c1cfd7080f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.170813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:34.194532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932e675-e855-4fdb-bd9e-b5cbd38dee88 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.228113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:34.232880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752f597-0f8e-4262-bec3-aee556d0a23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.233552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752f597-0f8e-4262-bec3-aee556d0a23c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.233836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752f597-0f8e-4262-bec3-aee556d0a23c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.234115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752f597-0f8e-4262-bec3-aee556d0a23c None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.234950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752f597-0f8e-4262-bec3-aee556d0a23c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:34.235272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752f597-0f8e-4262-bec3-aee556d0a23c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.236800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752f597-0f8e-4262-bec3-aee556d0a23c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.249885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 256/539] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:34.255772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752f597-0f8e-4262-bec3-aee556d0a23c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.360968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752f597-0f8e-4262-bec3-aee556d0a23c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.396631 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42d7af-1072-4539-b3fb-cbdc26853cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.397298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42d7af-1072-4539-b3fb-cbdc26853cb6 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.397586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42d7af-1072-4539-b3fb-cbdc26853cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.397866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42d7af-1072-4539-b3fb-cbdc26853cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.398781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42d7af-1072-4539-b3fb-cbdc26853cb6 None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=6640e4c799514429a2fe7850369af4df) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:34.399119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42d7af-1072-4539-b3fb-cbdc26853cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.400544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42d7af-1072-4539-b3fb-cbdc26853cb6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.407184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 257/540] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:34 2022] GET /identity/v3/role_assignments?user.id=6640e4c799514429a2fe7850369af4df&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:34.423427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ce08a-5b40-4128-91a5-e4c1cfd7080f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.486271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6449bd3-9c7d-448c-bc74-0752d34a2eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.486990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6449bd3-9c7d-448c-bc74-0752d34a2eea None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.487330 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6449bd3-9c7d-448c-bc74-0752d34a2eea None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.487660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6449bd3-9c7d-448c-bc74-0752d34a2eea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.488543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6449bd3-9c7d-448c-bc74-0752d34a2eea None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=ecd61347f8874b89aada19772ec56179) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:34.488855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6449bd3-9c7d-448c-bc74-0752d34a2eea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.490419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6449bd3-9c7d-448c-bc74-0752d34a2eea None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.503097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 284/541] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:34 2022] GET /identity/v3/role_assignments?user.id=ecd61347f8874b89aada19772ec56179&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:34.648037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e630df4-a9ed-4ddc-913d-3c9602e31ed7 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.648973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e630df4-a9ed-4ddc-913d-3c9602e31ed7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.649447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e630df4-a9ed-4ddc-913d-3c9602e31ed7 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.651061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 258/542] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:34.666149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59784893-cc42-43ce-bbf3-c833e47e6f30 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.666512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59784893-cc42-43ce-bbf3-c833e47e6f30 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.666791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59784893-cc42-43ce-bbf3-c833e47e6f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.674915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028e7c7-b08f-4adf-a78e-70fcd9257833 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.675229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028e7c7-b08f-4adf-a78e-70fcd9257833 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.675536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028e7c7-b08f-4adf-a78e-70fcd9257833 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.677289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 259/543] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:34.682861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e575-c323-4000-b9e4-fd4407a55fc1 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.683139 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e575-c323-4000-b9e4-fd4407a55fc1 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.683420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6e575-c323-4000-b9e4-fd4407a55fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.696463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-59784893-cc42-43ce-bbf3-c833e47e6f30 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:34.697672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-59784893-cc42-43ce-bbf3-c833e47e6f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:34.701978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-7ed6e575-c323-4000-b9e4-fd4407a55fc1 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:34.703366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7ed6e575-c323-4000-b9e4-fd4407a55fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:34.782173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:34.782918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 260/544] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:34.804969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6e575-c323-4000-b9e4-fd4407a55fc1 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.810024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 285/545] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:34.867202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59784893-cc42-43ce-bbf3-c833e47e6f30 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.869527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:34.952867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6e575-c323-4000-b9e4-fd4407a55fc1 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.019121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11639675-1129-465b-a98c-1e52568ebcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.019923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11639675-1129-465b-a98c-1e52568ebcfa None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.020201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11639675-1129-465b-a98c-1e52568ebcfa None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.020471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11639675-1129-465b-a98c-1e52568ebcfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.032924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59784893-cc42-43ce-bbf3-c833e47e6f30 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.044563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11639675-1129-465b-a98c-1e52568ebcfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-11639675-1129-465b-a98c-1e52568ebcfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:35.052343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:35.053444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:35.054374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 261/546] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:35.060966 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11639675-1129-465b-a98c-1e52568ebcfa None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:35.098394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db196d-0564-4f2a-a2af-2c8a3ed46723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.099228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db196d-0564-4f2a-a2af-2c8a3ed46723 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.099509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db196d-0564-4f2a-a2af-2c8a3ed46723 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.099781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db196d-0564-4f2a-a2af-2c8a3ed46723 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.111014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db196d-0564-4f2a-a2af-2c8a3ed46723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:35.115595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-e7db196d-0564-4f2a-a2af-2c8a3ed46723 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:35.118838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:35.119765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:35.120699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 286/547] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:35.127391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db196d-0564-4f2a-a2af-2c8a3ed46723 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:35.177733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11639675-1129-465b-a98c-1e52568ebcfa None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.213271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cb261-dfd5-4017-a287-e7a4150b490e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.214835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cb261-dfd5-4017-a287-e7a4150b490e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.215727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cb261-dfd5-4017-a287-e7a4150b490e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.216256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cb261-dfd5-4017-a287-e7a4150b490e None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.217186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cb261-dfd5-4017-a287-e7a4150b490e None admin] RBAC: Adding query filter params (name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:35.217666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cb261-dfd5-4017-a287-e7a4150b490e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:35.219246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cb261-dfd5-4017-a287-e7a4150b490e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:35.228600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 262/548] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:35.234109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cb261-dfd5-4017-a287-e7a4150b490e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:35.243884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:35.341600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cb261-dfd5-4017-a287-e7a4150b490e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.351245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:35.352622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db196d-0564-4f2a-a2af-2c8a3ed46723 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.377211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13325c42-b99a-47c2-aa78-ae6a38e13543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.378308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13325c42-b99a-47c2-aa78-ae6a38e13543 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.378807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13325c42-b99a-47c2-aa78-ae6a38e13543 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.379336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13325c42-b99a-47c2-aa78-ae6a38e13543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.391383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13325c42-b99a-47c2-aa78-ae6a38e13543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:35.393177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13325c42-b99a-47c2-aa78-ae6a38e13543 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-13325c42-b99a-47c2-aa78-ae6a38e13543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:35.412323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:35.413253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:35.414150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:35.414150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:35.414150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:35.414150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:35.416451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 263/549] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:35.421963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13325c42-b99a-47c2-aa78-ae6a38e13543 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:35.455821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054df21-144d-4192-89ca-85d9179743a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.456476 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054df21-144d-4192-89ca-85d9179743a0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.456766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054df21-144d-4192-89ca-85d9179743a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.457035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054df21-144d-4192-89ca-85d9179743a0 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.457790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054df21-144d-4192-89ca-85d9179743a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:35.458095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054df21-144d-4192-89ca-85d9179743a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:35.459527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054df21-144d-4192-89ca-85d9179743a0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:35.480732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 287/550] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:35.486156 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054df21-144d-4192-89ca-85d9179743a0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:35.569183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13325c42-b99a-47c2-aa78-ae6a38e13543 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.634877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115d957-0684-465e-ad41-c16c229c1264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.635545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d957-0684-465e-ad41-c16c229c1264 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.635845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d957-0684-465e-ad41-c16c229c1264 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.636651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d957-0684-465e-ad41-c16c229c1264 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.637439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115d957-0684-465e-ad41-c16c229c1264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:35.637745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115d957-0684-465e-ad41-c16c229c1264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:35.639458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115d957-0684-465e-ad41-c16c229c1264 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:35.652292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 264/551] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:35.657895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115d957-0684-465e-ad41-c16c229c1264 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:35.680624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:35.709183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054df21-144d-4192-89ca-85d9179743a0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.734228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:35.753174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0bd30-9d62-4112-a508-ceb686e625d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.753988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0bd30-9d62-4112-a508-ceb686e625d9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.754275 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0bd30-9d62-4112-a508-ceb686e625d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.755412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0bd30-9d62-4112-a508-ceb686e625d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.774437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0bd30-9d62-4112-a508-ceb686e625d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:35.777016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0bd30-9d62-4112-a508-ceb686e625d9 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:35.789945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115d957-0684-465e-ad41-c16c229c1264 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-92e0bd30-9d62-4112-a508-ceb686e625d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:35.814117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:35.815098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:35.815964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:35.815964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:35.815964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:35.815964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:35.816718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 288/552] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:35.821601 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0bd30-9d62-4112-a508-ceb686e625d9 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:35.847687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc34641-d2f0-4dcb-9196-b63817041a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.848499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc34641-d2f0-4dcb-9196-b63817041a6d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.848837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc34641-d2f0-4dcb-9196-b63817041a6d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.849120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc34641-d2f0-4dcb-9196-b63817041a6d None admin] PATH_INFO: `/v3/users/nova` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.865399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc34641-d2f0-4dcb-9196-b63817041a6d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:35.866940 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc34641-d2f0-4dcb-9196-b63817041a6d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-8bc34641-d2f0-4dcb-9196-b63817041a6d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:35.906366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:35.907464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:35.908411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:35.908411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:35.908411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:35.908411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:35.908411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:35.908411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:35.908411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:35.908411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:35.908411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:40:35.908411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:35.910503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 265/553] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 05:40:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:35.915919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc34641-d2f0-4dcb-9196-b63817041a6d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.022116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0bd30-9d62-4112-a508-ceb686e625d9 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.074844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:36.077483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ec844-efe4-4dae-aab2-653bd48329fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.078146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ec844-efe4-4dae-aab2-653bd48329fe None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.078426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ec844-efe4-4dae-aab2-653bd48329fe None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.078741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ec844-efe4-4dae-aab2-653bd48329fe None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.079456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ec844-efe4-4dae-aab2-653bd48329fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:36.079761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ec844-efe4-4dae-aab2-653bd48329fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.081150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ec844-efe4-4dae-aab2-653bd48329fe None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.092786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc34641-d2f0-4dcb-9196-b63817041a6d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.095058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 289/554] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:36.096445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f29ee1-a462-4f1e-b5cc-73e693fcfe6f None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.096731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f29ee1-a462-4f1e-b5cc-73e693fcfe6f None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.097004 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f29ee1-a462-4f1e-b5cc-73e693fcfe6f None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.102959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 290/555] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:36.103801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f29ee1-a462-4f1e-b5cc-73e693fcfe6f None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.134204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14623a81-acf4-42fe-8a05-8008e7d7f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.135252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14623a81-acf4-42fe-8a05-8008e7d7f253 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.135821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14623a81-acf4-42fe-8a05-8008e7d7f253 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.136107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14623a81-acf4-42fe-8a05-8008e7d7f253 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.137188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14623a81-acf4-42fe-8a05-8008e7d7f253 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:36.137508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14623a81-acf4-42fe-8a05-8008e7d7f253 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.140172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14623a81-acf4-42fe-8a05-8008e7d7f253 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.154861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:36.162425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 266/556] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 05:40:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:36.164317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfc65d-159e-4ae6-899e-0a1f3f541206 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.164317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfc65d-159e-4ae6-899e-0a1f3f541206 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.164470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfc65d-159e-4ae6-899e-0a1f3f541206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.190626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-89bfc65d-159e-4ae6-899e-0a1f3f541206 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:36.192101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-89bfc65d-159e-4ae6-899e-0a1f3f541206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:36.238242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f29ee1-a462-4f1e-b5cc-73e693fcfe6f None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.282113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 267/557] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:36.283123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfc65d-159e-4ae6-899e-0a1f3f541206 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.286577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe96dcf-7c44-4434-8a16-065da04292aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.287386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe96dcf-7c44-4434-8a16-065da04292aa None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.288686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe96dcf-7c44-4434-8a16-065da04292aa None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.288978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe96dcf-7c44-4434-8a16-065da04292aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.298404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe96dcf-7c44-4434-8a16-065da04292aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.299929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe96dcf-7c44-4434-8a16-065da04292aa None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-2fe96dcf-7c44-4434-8a16-065da04292aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:36.324811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:36.326991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 291/558] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:36.328970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe96dcf-7c44-4434-8a16-065da04292aa None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.389231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:36.426969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfc65d-159e-4ae6-899e-0a1f3f541206 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.459568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:36.469617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe96dcf-7c44-4434-8a16-065da04292aa None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.472231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129a3c2-ccf4-4654-92cd-3fed2fc1ff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.473323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129a3c2-ccf4-4654-92cd-3fed2fc1ff0c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.473323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129a3c2-ccf4-4654-92cd-3fed2fc1ff0c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.473572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129a3c2-ccf4-4654-92cd-3fed2fc1ff0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.486094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129a3c2-ccf4-4654-92cd-3fed2fc1ff0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.488259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129a3c2-ccf4-4654-92cd-3fed2fc1ff0c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.513070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce175387-6254-44ce-b708-f9a41263f796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.514141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce175387-6254-44ce-b708-f9a41263f796 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.514419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce175387-6254-44ce-b708-f9a41263f796 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.514738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce175387-6254-44ce-b708-f9a41263f796 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-8129a3c2-ccf4-4654-92cd-3fed2fc1ff0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:36.516477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:36.517595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:36.518730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:36.518730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:36.518730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:36.518730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:36.525915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 268/559] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:36.527878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129a3c2-ccf4-4654-92cd-3fed2fc1ff0c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.534166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce175387-6254-44ce-b708-f9a41263f796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-ce175387-6254-44ce-b708-f9a41263f796 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:36.543099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:36.544245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:36.545287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:36.550568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 292/560] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:36.551616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce175387-6254-44ce-b708-f9a41263f796 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.669478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce175387-6254-44ce-b708-f9a41263f796 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.673687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129a3c2-ccf4-4654-92cd-3fed2fc1ff0c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.703388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:36.709956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281767c-24ed-458c-9574-8b67fd98cae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.711234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281767c-24ed-458c-9574-8b67fd98cae8 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.711517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281767c-24ed-458c-9574-8b67fd98cae8 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.711790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281767c-24ed-458c-9574-8b67fd98cae8 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.712589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281767c-24ed-458c-9574-8b67fd98cae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:36.712985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281767c-24ed-458c-9574-8b67fd98cae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.715757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281767c-24ed-458c-9574-8b67fd98cae8 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.721734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9043d1a-76e2-4141-be04-611dd384d16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.722413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9043d1a-76e2-4141-be04-611dd384d16e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.722736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9043d1a-76e2-4141-be04-611dd384d16e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.723009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9043d1a-76e2-4141-be04-611dd384d16e None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.725373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9043d1a-76e2-4141-be04-611dd384d16e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:36.725373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9043d1a-76e2-4141-be04-611dd384d16e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.725834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9043d1a-76e2-4141-be04-611dd384d16e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.738799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 293/561] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:36.740943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281767c-24ed-458c-9574-8b67fd98cae8 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.759103 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 269/562] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:36.760027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9043d1a-76e2-4141-be04-611dd384d16e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.767935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:36.878682 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281767c-24ed-458c-9574-8b67fd98cae8 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.883461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9043d1a-76e2-4141-be04-611dd384d16e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.916418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d577c0-22c3-47c2-a2d3-113f55714856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.917118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d577c0-22c3-47c2-a2d3-113f55714856 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.917403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d577c0-22c3-47c2-a2d3-113f55714856 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.917680 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d577c0-22c3-47c2-a2d3-113f55714856 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.918427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d577c0-22c3-47c2-a2d3-113f55714856 None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:36.918768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d577c0-22c3-47c2-a2d3-113f55714856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.920138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d577c0-22c3-47c2-a2d3-113f55714856 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.920597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9e6d8-4837-441b-869f-784d940bf735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.921386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9e6d8-4837-441b-869f-784d940bf735 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.921664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9e6d8-4837-441b-869f-784d940bf735 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.921936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9e6d8-4837-441b-869f-784d940bf735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.928117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 294/563] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:36.928555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9e6d8-4837-441b-869f-784d940bf735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.928900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d577c0-22c3-47c2-a2d3-113f55714856 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.930064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9e6d8-4837-441b-869f-784d940bf735 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-ead9e6d8-4837-441b-869f-784d940bf735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:36.944890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:36.945936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:36.947031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:36.947031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:36.947031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:36.947031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:36.947031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:36.947031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:36.947031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:36.947031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:36.947031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 270/564] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:36.947643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca6a83-2b6a-433b-b839-c003ba4e1051 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.947643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca6a83-2b6a-433b-b839-c003ba4e1051 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.947822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca6a83-2b6a-433b-b839-c003ba4e1051 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.949786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 271/565] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:36.951756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f8eb-6c56-4f98-a9f9-79442dc9158a None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.952046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f8eb-6c56-4f98-a9f9-79442dc9158a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.952325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f8eb-6c56-4f98-a9f9-79442dc9158a None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.953680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 272/566] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:36.954241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4f8eb-6c56-4f98-a9f9-79442dc9158a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.994304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.015209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.035933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d577c0-22c3-47c2-a2d3-113f55714856 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.054748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4f8eb-6c56-4f98-a9f9-79442dc9158a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.068916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2db23-7e58-4d1b-aa62-ebd232b42476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.069561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2db23-7e58-4d1b-aa62-ebd232b42476 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.069842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2db23-7e58-4d1b-aa62-ebd232b42476 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.070117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2db23-7e58-4d1b-aa62-ebd232b42476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.088207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00ae6c-bcb2-4cf8-b657-042c92dada88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.089071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00ae6c-bcb2-4cf8-b657-042c92dada88 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.089355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00ae6c-bcb2-4cf8-b657-042c92dada88 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.089632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00ae6c-bcb2-4cf8-b657-042c92dada88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.099934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00ae6c-bcb2-4cf8-b657-042c92dada88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:37.101414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00ae6c-bcb2-4cf8-b657-042c92dada88 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-7a00ae6c-bcb2-4cf8-b657-042c92dada88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:37.118318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:37.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:37.121667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:37.121667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:37.121667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:37.121667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:37.121667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 273/567] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:37.121667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00ae6c-bcb2-4cf8-b657-042c92dada88 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.137995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2db23-7e58-4d1b-aa62-ebd232b42476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:37.139703 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2db23-7e58-4d1b-aa62-ebd232b42476 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:37.207439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.214652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.217114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00ae6c-bcb2-4cf8-b657-042c92dada88 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.235846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 295/568] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:37.237117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158f031-1c10-4ac2-b8ed-624331dcf726 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.237409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158f031-1c10-4ac2-b8ed-624331dcf726 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.237679 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158f031-1c10-4ac2-b8ed-624331dcf726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.253002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-5158f031-1c10-4ac2-b8ed-624331dcf726 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:37.254231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5158f031-1c10-4ac2-b8ed-624331dcf726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:37.254359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e0514-a1b1-4732-874a-b5f407e4fce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.254990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e0514-a1b1-4732-874a-b5f407e4fce8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.255272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e0514-a1b1-4732-874a-b5f407e4fce8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.255544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e0514-a1b1-4732-874a-b5f407e4fce8 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.256324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e0514-a1b1-4732-874a-b5f407e4fce8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:37.256632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e0514-a1b1-4732-874a-b5f407e4fce8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:37.258050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e0514-a1b1-4732-874a-b5f407e4fce8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:37.269596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 274/569] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:37.270892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031947e-5cf7-41f3-9821-6d17a7351c99 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.271179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031947e-5cf7-41f3-9821-6d17a7351c99 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.271455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031947e-5cf7-41f3-9821-6d17a7351c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.287499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c031947e-5cf7-41f3-9821-6d17a7351c99 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:37.288679 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c031947e-5cf7-41f3-9821-6d17a7351c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:37.319972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 296/570] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:37.320785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158f031-1c10-4ac2-b8ed-624331dcf726 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.356273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 275/571] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:37.357272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031947e-5cf7-41f3-9821-6d17a7351c99 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.435233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.449555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158f031-1c10-4ac2-b8ed-624331dcf726 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.454311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031947e-5cf7-41f3-9821-6d17a7351c99 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.458612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.482427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93458c85-a84f-4426-ac2f-b9e3c842c082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.483130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458c85-a84f-4426-ac2f-b9e3c842c082 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.483416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458c85-a84f-4426-ac2f-b9e3c842c082 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.483689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458c85-a84f-4426-ac2f-b9e3c842c082 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.484404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93458c85-a84f-4426-ac2f-b9e3c842c082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:37.484751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93458c85-a84f-4426-ac2f-b9e3c842c082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:37.486077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93458c85-a84f-4426-ac2f-b9e3c842c082 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:37.487994 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa7caa-e3b0-44f7-b4f4-87d6e75fa3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.488775 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa7caa-e3b0-44f7-b4f4-87d6e75fa3f1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.489055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa7caa-e3b0-44f7-b4f4-87d6e75fa3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.489327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa7caa-e3b0-44f7-b4f4-87d6e75fa3f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.498235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 297/572] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:37.499170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93458c85-a84f-4426-ac2f-b9e3c842c082 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.499276 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa7caa-e3b0-44f7-b4f4-87d6e75fa3f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:37.500713 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa7caa-e3b0-44f7-b4f4-87d6e75fa3f1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-3bfa7caa-e3b0-44f7-b4f4-87d6e75fa3f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:37.519787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:37.520896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:37.522068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:37.522068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:37.522068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:37.522068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:37.522068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 276/573] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:37.522068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa7caa-e3b0-44f7-b4f4-87d6e75fa3f1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.629355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa7caa-e3b0-44f7-b4f4-87d6e75fa3f1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.630579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93458c85-a84f-4426-ac2f-b9e3c842c082 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.659507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.663782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cb36e-536f-4c55-93dd-ea5f5c545c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.664047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59273d0b-2a4a-439c-9654-8309691ff0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.664667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cb36e-536f-4c55-93dd-ea5f5c545c2e None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.664831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59273d0b-2a4a-439c-9654-8309691ff0cc None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.664943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cb36e-536f-4c55-93dd-ea5f5c545c2e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.665110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59273d0b-2a4a-439c-9654-8309691ff0cc None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.665201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cb36e-536f-4c55-93dd-ea5f5c545c2e None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/067b827625e0427294b93b027597a5f3/roles/9cc15ca9e61f425cacb9c1a9f52073dd` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.665386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59273d0b-2a4a-439c-9654-8309691ff0cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.675143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59273d0b-2a4a-439c-9654-8309691ff0cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:37.679390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cb36e-536f-4c55-93dd-ea5f5c545c2e None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=067b827625e0427294b93b027597a5f3, role_id=9cc15ca9e61f425cacb9c1a9f52073dd)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:37.680869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cb36e-536f-4c55-93dd-ea5f5c545c2e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-59273d0b-2a4a-439c-9654-8309691ff0cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:37.682964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:37.683878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:37.684722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 277/574] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:37.685475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59273d0b-2a4a-439c-9654-8309691ff0cc None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.755563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 298/575] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:37 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/067b827625e0427294b93b027597a5f3/roles/9cc15ca9e61f425cacb9c1a9f52073dd => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:37.756435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cb36e-536f-4c55-93dd-ea5f5c545c2e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.777633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.796349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59273d0b-2a4a-439c-9654-8309691ff0cc None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.835567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf59fd-6346-4b72-84c9-89430dafae5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.836392 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf59fd-6346-4b72-84c9-89430dafae5f None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.836680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf59fd-6346-4b72-84c9-89430dafae5f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.836961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf59fd-6346-4b72-84c9-89430dafae5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.849034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf59fd-6346-4b72-84c9-89430dafae5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-8bcf59fd-6346-4b72-84c9-89430dafae5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:37.857233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:37.858298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:37.859190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 278/576] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:37.859915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf59fd-6346-4b72-84c9-89430dafae5f None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.873481 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cb36e-536f-4c55-93dd-ea5f5c545c2e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.919422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958d0b5-196f-48b0-b913-caba00736262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.920230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958d0b5-196f-48b0-b913-caba00736262 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.920513 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958d0b5-196f-48b0-b913-caba00736262 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.921337 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958d0b5-196f-48b0-b913-caba00736262 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.924492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.928485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958d0b5-196f-48b0-b913-caba00736262 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:37.929946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958d0b5-196f-48b0-b913-caba00736262 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-6958d0b5-196f-48b0-b913-caba00736262 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:37.945499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:37.946779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 299/577] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:37.951749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958d0b5-196f-48b0-b913-caba00736262 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.978592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf59fd-6346-4b72-84c9-89430dafae5f None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.012743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8589e31-68a5-4981-8da0-69c7ca80a1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.013410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8589e31-68a5-4981-8da0-69c7ca80a1c0 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.013937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8589e31-68a5-4981-8da0-69c7ca80a1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.014227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8589e31-68a5-4981-8da0-69c7ca80a1c0 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.014985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8589e31-68a5-4981-8da0-69c7ca80a1c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:38.015286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8589e31-68a5-4981-8da0-69c7ca80a1c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.016672 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8589e31-68a5-4981-8da0-69c7ca80a1c0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.030559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 279/578] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:38.031743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8589e31-68a5-4981-8da0-69c7ca80a1c0 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.038949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.060386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958d0b5-196f-48b0-b913-caba00736262 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.097764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a7637-cbe0-4e26-bca2-7f3546d7947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.098440 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a7637-cbe0-4e26-bca2-7f3546d7947a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.098764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a7637-cbe0-4e26-bca2-7f3546d7947a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.099030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a7637-cbe0-4e26-bca2-7f3546d7947a None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.099776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a7637-cbe0-4e26-bca2-7f3546d7947a None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:38.100083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a7637-cbe0-4e26-bca2-7f3546d7947a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.101475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a7637-cbe0-4e26-bca2-7f3546d7947a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.109948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 300/579] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:38.110569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a7637-cbe0-4e26-bca2-7f3546d7947a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.140050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8589e31-68a5-4981-8da0-69c7ca80a1c0 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.153728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.175434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930df5cb-6aaf-4369-9b50-736a5cc95de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.176103 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df5cb-6aaf-4369-9b50-736a5cc95de5 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.176382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df5cb-6aaf-4369-9b50-736a5cc95de5 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.176652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df5cb-6aaf-4369-9b50-736a5cc95de5 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.177383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930df5cb-6aaf-4369-9b50-736a5cc95de5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:38.177685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930df5cb-6aaf-4369-9b50-736a5cc95de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.179171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930df5cb-6aaf-4369-9b50-736a5cc95de5 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.193600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 280/580] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:38.194772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930df5cb-6aaf-4369-9b50-736a5cc95de5 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.219478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a7637-cbe0-4e26-bca2-7f3546d7947a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.255016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5ad9b-b16c-4943-aedb-3f6a5b98a224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.256025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ad9b-b16c-4943-aedb-3f6a5b98a224 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.256530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ad9b-b16c-4943-aedb-3f6a5b98a224 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.257033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ad9b-b16c-4943-aedb-3f6a5b98a224 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.258081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5ad9b-b16c-4943-aedb-3f6a5b98a224 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:38.258633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5ad9b-b16c-4943-aedb-3f6a5b98a224 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.260198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5ad9b-b16c-4943-aedb-3f6a5b98a224 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.271425 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.283902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 301/581] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:38.283902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5ad9b-b16c-4943-aedb-3f6a5b98a224 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.306745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930df5cb-6aaf-4369-9b50-736a5cc95de5 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.341417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f800aa8-dbf5-4cec-bfb7-8eccbbca59e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.342208 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f800aa8-dbf5-4cec-bfb7-8eccbbca59e3 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.342520 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f800aa8-dbf5-4cec-bfb7-8eccbbca59e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.342786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f800aa8-dbf5-4cec-bfb7-8eccbbca59e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.349691 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f800aa8-dbf5-4cec-bfb7-8eccbbca59e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.351168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f800aa8-dbf5-4cec-bfb7-8eccbbca59e3 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-7f800aa8-dbf5-4cec-bfb7-8eccbbca59e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:38.366448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:38.367398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:38.368329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:38.368329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:38.368329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:38.368329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:38.368329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:38.368329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:38.368329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:38.368329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:38.368329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 281/582] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:38.368845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f800aa8-dbf5-4cec-bfb7-8eccbbca59e3 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.397066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5ad9b-b16c-4943-aedb-3f6a5b98a224 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.406361 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.434533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb770ce0-c004-49ee-a0e2-e3a46afdb9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.435335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb770ce0-c004-49ee-a0e2-e3a46afdb9cd None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.435616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb770ce0-c004-49ee-a0e2-e3a46afdb9cd None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.435892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb770ce0-c004-49ee-a0e2-e3a46afdb9cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.446973 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb770ce0-c004-49ee-a0e2-e3a46afdb9cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.448444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb770ce0-c004-49ee-a0e2-e3a46afdb9cd None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-fb770ce0-c004-49ee-a0e2-e3a46afdb9cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:38.468025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:38.468942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:38.469885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:38.469885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:38.469885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:38.469885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:38.469885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 302/583] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:38.470181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb770ce0-c004-49ee-a0e2-e3a46afdb9cd None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.478808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f800aa8-dbf5-4cec-bfb7-8eccbbca59e3 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.515968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3feab4d-0824-4d0f-b4ac-b9bcc5a22f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.516767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3feab4d-0824-4d0f-b4ac-b9bcc5a22f2a None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.517048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3feab4d-0824-4d0f-b4ac-b9bcc5a22f2a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.517324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3feab4d-0824-4d0f-b4ac-b9bcc5a22f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.527928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3feab4d-0824-4d0f-b4ac-b9bcc5a22f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.529398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3feab4d-0824-4d0f-b4ac-b9bcc5a22f2a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-b3feab4d-0824-4d0f-b4ac-b9bcc5a22f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:38.548030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:38.548971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:38.549864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:38.549864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:38.549864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:38.549864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:38.549864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 282/584] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:38.550153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3feab4d-0824-4d0f-b4ac-b9bcc5a22f2a None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.558709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.607965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb770ce0-c004-49ee-a0e2-e3a46afdb9cd None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.639371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.641456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8d53c-0beb-43f3-a218-131c844d348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.642418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d53c-0beb-43f3-a218-131c844d348a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.642800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d53c-0beb-43f3-a218-131c844d348a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.643143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d53c-0beb-43f3-a218-131c844d348a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.654111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8d53c-0beb-43f3-a218-131c844d348a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.655596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8d53c-0beb-43f3-a218-131c844d348a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.656816 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3feab4d-0824-4d0f-b4ac-b9bcc5a22f2a None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5ee8d53c-0beb-43f3-a218-131c844d348a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:38.673157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:38.674276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:38.675434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:38.675434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:38.675434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:38.675434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:38.675434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 303/585] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:38.675725 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8d53c-0beb-43f3-a218-131c844d348a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.697778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd1855-7e49-4d3f-bfa5-5389cc0cbcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.698486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd1855-7e49-4d3f-bfa5-5389cc0cbcd5 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.698851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd1855-7e49-4d3f-bfa5-5389cc0cbcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.699132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd1855-7e49-4d3f-bfa5-5389cc0cbcd5 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.699910 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd1855-7e49-4d3f-bfa5-5389cc0cbcd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:38.700221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd1855-7e49-4d3f-bfa5-5389cc0cbcd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.701641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd1855-7e49-4d3f-bfa5-5389cc0cbcd5 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.714898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 283/586] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:38.720319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd1855-7e49-4d3f-bfa5-5389cc0cbcd5 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.781767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8d53c-0beb-43f3-a218-131c844d348a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.783264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.824140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f34206-370d-4cbc-8405-aa4744d5fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.824809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f34206-370d-4cbc-8405-aa4744d5fa1a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.825091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f34206-370d-4cbc-8405-aa4744d5fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.825368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f34206-370d-4cbc-8405-aa4744d5fa1a None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.825906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd1855-7e49-4d3f-bfa5-5389cc0cbcd5 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.826085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f34206-370d-4cbc-8405-aa4744d5fa1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:38.826394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f34206-370d-4cbc-8405-aa4744d5fa1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.827794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f34206-370d-4cbc-8405-aa4744d5fa1a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.839597 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 304/587] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:38.840745 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f34206-370d-4cbc-8405-aa4744d5fa1a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.871584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c286b-9e9f-4c62-a1c5-9cdbda91744d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.872251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c286b-9e9f-4c62-a1c5-9cdbda91744d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.872531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c286b-9e9f-4c62-a1c5-9cdbda91744d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.872873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c286b-9e9f-4c62-a1c5-9cdbda91744d None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.873513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c286b-9e9f-4c62-a1c5-9cdbda91744d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:38.873985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c286b-9e9f-4c62-a1c5-9cdbda91744d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.875203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c286b-9e9f-4c62-a1c5-9cdbda91744d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.889216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 284/588] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:38.889865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c286b-9e9f-4c62-a1c5-9cdbda91744d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.899076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.961063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f34206-370d-4cbc-8405-aa4744d5fa1a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.997743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e5698-4dd1-44a1-92b8-92f733c92cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.998333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e5698-4dd1-44a1-92b8-92f733c92cec None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.998647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e5698-4dd1-44a1-92b8-92f733c92cec None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.998922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e5698-4dd1-44a1-92b8-92f733c92cec None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.999021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c286b-9e9f-4c62-a1c5-9cdbda91744d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.999639 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e5698-4dd1-44a1-92b8-92f733c92cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:38.999944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e5698-4dd1-44a1-92b8-92f733c92cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.001780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e5698-4dd1-44a1-92b8-92f733c92cec None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.013534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 305/589] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:39.014655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e5698-4dd1-44a1-92b8-92f733c92cec None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.038616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4c3c4-3e1f-445f-b429-c0e71754b0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.038850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.039275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4c3c4-3e1f-445f-b429-c0e71754b0be None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.039562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4c3c4-3e1f-445f-b429-c0e71754b0be None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.039842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4c3c4-3e1f-445f-b429-c0e71754b0be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.040667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4c3c4-3e1f-445f-b429-c0e71754b0be None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=5c359fc0168141fb8da3f953884fd2e5) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:39.041038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4c3c4-3e1f-445f-b429-c0e71754b0be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.042964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4c3c4-3e1f-445f-b429-c0e71754b0be None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.058499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 285/590] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/role_assignments?user.id=5c359fc0168141fb8da3f953884fd2e5&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:39.058499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4c3c4-3e1f-445f-b429-c0e71754b0be None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.135324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e5698-4dd1-44a1-92b8-92f733c92cec None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.154574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.174174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0541e-e5b3-4332-9eb9-df58ae6da429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.175020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0541e-e5b3-4332-9eb9-df58ae6da429 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.175301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0541e-e5b3-4332-9eb9-df58ae6da429 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.180569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0541e-e5b3-4332-9eb9-df58ae6da429 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.185240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0541e-e5b3-4332-9eb9-df58ae6da429 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.186745 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0541e-e5b3-4332-9eb9-df58ae6da429 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.199259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4c3c4-3e1f-445f-b429-c0e71754b0be None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-76c0541e-e5b3-4332-9eb9-df58ae6da429 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:39.202496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:39.203462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 306/591] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:39.204866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0541e-e5b3-4332-9eb9-df58ae6da429 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.248885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776833-f0ba-4636-ba90-c7a00950b440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.250261 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776833-f0ba-4636-ba90-c7a00950b440 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.250632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776833-f0ba-4636-ba90-c7a00950b440 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.251073 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776833-f0ba-4636-ba90-c7a00950b440 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.259415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776833-f0ba-4636-ba90-c7a00950b440 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.261036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776833-f0ba-4636-ba90-c7a00950b440 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-a0776833-f0ba-4636-ba90-c7a00950b440 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:39.280802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:39.281809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:39.282784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:39.282784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:39.282784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:39.282784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:39.282784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:39.282784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:39.282784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:39.282784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:39.282784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:40:39.282784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:39.283888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 286/592] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:39.285013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776833-f0ba-4636-ba90-c7a00950b440 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.309579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.329849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0541e-e5b3-4332-9eb9-df58ae6da429 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.372114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe40c58-c7a3-4ac5-be3b-f530b510d109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.373076 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe40c58-c7a3-4ac5-be3b-f530b510d109 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.373464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe40c58-c7a3-4ac5-be3b-f530b510d109 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.373841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe40c58-c7a3-4ac5-be3b-f530b510d109 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.381216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe40c58-c7a3-4ac5-be3b-f530b510d109 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.382844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe40c58-c7a3-4ac5-be3b-f530b510d109 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-1fe40c58-c7a3-4ac5-be3b-f530b510d109 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:39.397848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:39.398941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:39.399861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:39.399861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:39.399861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:39.399861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:39.399861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:39.399861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:39.399861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:39.399861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:39.399861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 307/593] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:39.400887 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe40c58-c7a3-4ac5-be3b-f530b510d109 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.407634 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776833-f0ba-4636-ba90-c7a00950b440 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.437418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.444021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d586817-993b-4919-9865-57273be45664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.444902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d586817-993b-4919-9865-57273be45664 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.445352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d586817-993b-4919-9865-57273be45664 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.445771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d586817-993b-4919-9865-57273be45664 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.446904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d586817-993b-4919-9865-57273be45664 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:39.447357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d586817-993b-4919-9865-57273be45664 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.448985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d586817-993b-4919-9865-57273be45664 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.471969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 287/594] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:39.473134 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d586817-993b-4919-9865-57273be45664 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.516359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe40c58-c7a3-4ac5-be3b-f530b510d109 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.554291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a190c-686f-4ee6-aa41-895614c1230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.555102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a190c-686f-4ee6-aa41-895614c1230d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.555444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a190c-686f-4ee6-aa41-895614c1230d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.555784 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a190c-686f-4ee6-aa41-895614c1230d None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.556703 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a190c-686f-4ee6-aa41-895614c1230d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:39.557072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a190c-686f-4ee6-aa41-895614c1230d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.558708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a190c-686f-4ee6-aa41-895614c1230d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.577634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 308/595] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:39.578783 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a190c-686f-4ee6-aa41-895614c1230d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.582779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.588389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d586817-993b-4919-9865-57273be45664 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.626898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f6603-6b15-4af1-bccf-2b6c9e272d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.628042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6603-6b15-4af1-bccf-2b6c9e272d01 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.628539 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6603-6b15-4af1-bccf-2b6c9e272d01 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.628997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6603-6b15-4af1-bccf-2b6c9e272d01 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.630031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f6603-6b15-4af1-bccf-2b6c9e272d01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:39.630538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f6603-6b15-4af1-bccf-2b6c9e272d01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.632326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f6603-6b15-4af1-bccf-2b6c9e272d01 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.646183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 288/596] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:39.647394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f6603-6b15-4af1-bccf-2b6c9e272d01 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.683905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a190c-686f-4ee6-aa41-895614c1230d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.700194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.719708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b20109-42c6-4cae-a3b0-fddb29c4d009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.720932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b20109-42c6-4cae-a3b0-fddb29c4d009 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.720932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b20109-42c6-4cae-a3b0-fddb29c4d009 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.721120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b20109-42c6-4cae-a3b0-fddb29c4d009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.732292 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b20109-42c6-4cae-a3b0-fddb29c4d009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.733774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b20109-42c6-4cae-a3b0-fddb29c4d009 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-f3b20109-42c6-4cae-a3b0-fddb29c4d009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:39.751683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:39.752626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:39.753832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:39.753832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:39.753832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:39.753832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:39.753832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 309/597] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:39.753832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b20109-42c6-4cae-a3b0-fddb29c4d009 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.756637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f6603-6b15-4af1-bccf-2b6c9e272d01 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.794077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57098e-5728-4277-9f5f-f9b4d3334d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.794905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57098e-5728-4277-9f5f-f9b4d3334d21 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.795186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57098e-5728-4277-9f5f-f9b4d3334d21 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.795457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57098e-5728-4277-9f5f-f9b4d3334d21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.806969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57098e-5728-4277-9f5f-f9b4d3334d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.808563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57098e-5728-4277-9f5f-f9b4d3334d21 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.814001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-8a57098e-5728-4277-9f5f-f9b4d3334d21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:39.829019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:39.830091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:39.831062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:39.831062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:39.831062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:39.831062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:39.831062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 289/598] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:39.831062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57098e-5728-4277-9f5f-f9b4d3334d21 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.867082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b20109-42c6-4cae-a3b0-fddb29c4d009 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.906192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d355c-06d4-4a59-974f-67476fa7f3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.907477 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d355c-06d4-4a59-974f-67476fa7f3be None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.907969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d355c-06d4-4a59-974f-67476fa7f3be None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.908690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d355c-06d4-4a59-974f-67476fa7f3be None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/6e12361da5f849ee97607ca68ee6ef10/roles/9cc15ca9e61f425cacb9c1a9f52073dd` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.923441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d355c-06d4-4a59-974f-67476fa7f3be None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=6e12361da5f849ee97607ca68ee6ef10, role_id=9cc15ca9e61f425cacb9c1a9f52073dd)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.924976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d355c-06d4-4a59-974f-67476fa7f3be None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.943670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.946421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57098e-5728-4277-9f5f-f9b4d3334d21 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.979650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db04fa-eb3f-4556-b52c-5f4ef2347e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.980306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db04fa-eb3f-4556-b52c-5f4ef2347e6b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.980584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db04fa-eb3f-4556-b52c-5f4ef2347e6b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.980855 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db04fa-eb3f-4556-b52c-5f4ef2347e6b None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.981564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db04fa-eb3f-4556-b52c-5f4ef2347e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:39.981867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db04fa-eb3f-4556-b52c-5f4ef2347e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.983265 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db04fa-eb3f-4556-b52c-5f4ef2347e6b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.993891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 290/599] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:40.000906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3d59-4321-4d03-b349-219346245581 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.000906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3d59-4321-4d03-b349-219346245581 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.000906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3d59-4321-4d03-b349-219346245581 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.000906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 291/600] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:40.000906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3d59-4321-4d03-b349-219346245581 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.024336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 310/601] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:39 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/6e12361da5f849ee97607ca68ee6ef10/roles/9cc15ca9e61f425cacb9c1a9f52073dd => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:40.025153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d355c-06d4-4a59-974f-67476fa7f3be None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.150502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3d59-4321-4d03-b349-219346245581 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.153521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d355c-06d4-4a59-974f-67476fa7f3be None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.155324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.200973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf86a0c-a6e8-4d41-843e-1d7fa6291647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.201655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf86a0c-a6e8-4d41-843e-1d7fa6291647 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.202272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf86a0c-a6e8-4d41-843e-1d7fa6291647 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.202272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf86a0c-a6e8-4d41-843e-1d7fa6291647 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.203145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf86a0c-a6e8-4d41-843e-1d7fa6291647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:40.203345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa545c59-a813-4b1c-b4ab-2cc5097c762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.203492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf86a0c-a6e8-4d41-843e-1d7fa6291647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.204157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa545c59-a813-4b1c-b4ab-2cc5097c762b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.204437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa545c59-a813-4b1c-b4ab-2cc5097c762b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.204706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa545c59-a813-4b1c-b4ab-2cc5097c762b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.204815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf86a0c-a6e8-4d41-843e-1d7fa6291647 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.215493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa545c59-a813-4b1c-b4ab-2cc5097c762b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.217745 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa545c59-a813-4b1c-b4ab-2cc5097c762b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.220692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 292/602] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:40.221190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821614f0-ff7b-4618-82a5-0b0091e56fa6 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.221476 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821614f0-ff7b-4618-82a5-0b0091e56fa6 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.221750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821614f0-ff7b-4618-82a5-0b0091e56fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-fa545c59-a813-4b1c-b4ab-2cc5097c762b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:40.233021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:40.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:40.234985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:40.234985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:40.234985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:40.234985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:40.234985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:40.234985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:40.234985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:40.234985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:40.234985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 311/603] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:40.235717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa545c59-a813-4b1c-b4ab-2cc5097c762b None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.243086 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-821614f0-ff7b-4618-82a5-0b0091e56fa6 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:40.244564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-821614f0-ff7b-4618-82a5-0b0091e56fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:40.261970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.319628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 293/604] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:40.320294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821614f0-ff7b-4618-82a5-0b0091e56fa6 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.381150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa545c59-a813-4b1c-b4ab-2cc5097c762b None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.419842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f8054-688b-4bb0-835b-40fa170549f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.421053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f8054-688b-4bb0-835b-40fa170549f9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.421572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f8054-688b-4bb0-835b-40fa170549f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.422054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f8054-688b-4bb0-835b-40fa170549f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.423118 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821614f0-ff7b-4618-82a5-0b0091e56fa6 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.429729 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f8054-688b-4bb0-835b-40fa170549f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.431600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f8054-688b-4bb0-835b-40fa170549f9 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5b4f8054-688b-4bb0-835b-40fa170549f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:40.447384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:40.448837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:40.449741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:40.449741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:40.449741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:40.449741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:40.449741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:40.449741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:40.449741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:40.449741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:40.452401 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 312/605] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:40.453659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f8054-688b-4bb0-835b-40fa170549f9 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.458492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92702d66-2282-422f-93db-3889326b65d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.459167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92702d66-2282-422f-93db-3889326b65d8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.459502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92702d66-2282-422f-93db-3889326b65d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.459775 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92702d66-2282-422f-93db-3889326b65d8 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.460603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92702d66-2282-422f-93db-3889326b65d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:40.460913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92702d66-2282-422f-93db-3889326b65d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.462336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92702d66-2282-422f-93db-3889326b65d8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.475973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 294/606] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:40.476980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92702d66-2282-422f-93db-3889326b65d8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.483147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.490616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.565837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f8054-688b-4bb0-835b-40fa170549f9 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.590046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92702d66-2282-422f-93db-3889326b65d8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.605042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a7c83-5161-4bdd-8825-77dc0461295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.606195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7c83-5161-4bdd-8825-77dc0461295b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.606732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7c83-5161-4bdd-8825-77dc0461295b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.607263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7c83-5161-4bdd-8825-77dc0461295b None admin] PATH_INFO: `/v3/roles/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.618771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a7c83-5161-4bdd-8825-77dc0461295b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:40.625271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54517f6-10e0-469a-b3a8-66a206747305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.625938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54517f6-10e0-469a-b3a8-66a206747305 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.626268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54517f6-10e0-469a-b3a8-66a206747305 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.626600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54517f6-10e0-469a-b3a8-66a206747305 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.627440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54517f6-10e0-469a-b3a8-66a206747305 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:40.627748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54517f6-10e0-469a-b3a8-66a206747305 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5a7a7c83-5161-4bdd-8825-77dc0461295b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:40.628634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:40.629551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:40.630406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:40.631258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54517f6-10e0-469a-b3a8-66a206747305 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.633134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 313/607] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:40.634244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a7c83-5161-4bdd-8825-77dc0461295b None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.642981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 295/608] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:40.643817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54517f6-10e0-469a-b3a8-66a206747305 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.719821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.727641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.746483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a7c83-5161-4bdd-8825-77dc0461295b None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.752627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54517f6-10e0-469a-b3a8-66a206747305 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.785675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fb8bc-10a3-4215-bb89-8bda82e79038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.786959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fb8bc-10a3-4215-bb89-8bda82e79038 None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.787515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fb8bc-10a3-4215-bb89-8bda82e79038 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.788004 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fb8bc-10a3-4215-bb89-8bda82e79038 None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/6640e4c799514429a2fe7850369af4df/roles/9cc15ca9e61f425cacb9c1a9f52073dd` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.788209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a26e7-a4aa-4c8d-8151-1e370704ca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.788857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a26e7-a4aa-4c8d-8151-1e370704ca5d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.789163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a26e7-a4aa-4c8d-8151-1e370704ca5d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.789405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a26e7-a4aa-4c8d-8151-1e370704ca5d None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.790147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a26e7-a4aa-4c8d-8151-1e370704ca5d None admin] RBAC: Adding query filter params (name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:40.790451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a26e7-a4aa-4c8d-8151-1e370704ca5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.791887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a26e7-a4aa-4c8d-8151-1e370704ca5d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.801728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 296/609] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:40.802690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a26e7-a4aa-4c8d-8151-1e370704ca5d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.806987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fb8bc-10a3-4215-bb89-8bda82e79038 None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=6640e4c799514429a2fe7850369af4df, role_id=9cc15ca9e61f425cacb9c1a9f52073dd)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.809265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fb8bc-10a3-4215-bb89-8bda82e79038 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.898189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 314/610] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:40 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/6640e4c799514429a2fe7850369af4df/roles/9cc15ca9e61f425cacb9c1a9f52073dd => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:40.900888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fb8bc-10a3-4215-bb89-8bda82e79038 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.918294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a26e7-a4aa-4c8d-8151-1e370704ca5d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.985825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.998056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e04d4-16fd-4be9-aff8-83455f7b448b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.998987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e04d4-16fd-4be9-aff8-83455f7b448b None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.999265 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e04d4-16fd-4be9-aff8-83455f7b448b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.999533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e04d4-16fd-4be9-aff8-83455f7b448b None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/ecd61347f8874b89aada19772ec56179/roles/9cc15ca9e61f425cacb9c1a9f52073dd` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.032437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e04d4-16fd-4be9-aff8-83455f7b448b None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=ecd61347f8874b89aada19772ec56179, role_id=9cc15ca9e61f425cacb9c1a9f52073dd)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.033821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e04d4-16fd-4be9-aff8-83455f7b448b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.035672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fb8bc-10a3-4215-bb89-8bda82e79038 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.071227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.077133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e037b-706b-4c5a-8fdf-d54d33d82f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.077956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e037b-706b-4c5a-8fdf-d54d33d82f06 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.078238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e037b-706b-4c5a-8fdf-d54d33d82f06 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.078538 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e037b-706b-4c5a-8fdf-d54d33d82f06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.091455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e037b-706b-4c5a-8fdf-d54d33d82f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.093136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e037b-706b-4c5a-8fdf-d54d33d82f06 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-9d0e037b-706b-4c5a-8fdf-d54d33d82f06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:41.114084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:41.115090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:41.115986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:41.115986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:41.115986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:41.115986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:41.115986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 315/611] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:41.123467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e037b-706b-4c5a-8fdf-d54d33d82f06 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.128182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 297/612] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:40 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/ecd61347f8874b89aada19772ec56179/roles/9cc15ca9e61f425cacb9c1a9f52073dd => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:41.239969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8df566-d3ee-43ab-b1cd-7eea74a775bb None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.240269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8df566-d3ee-43ab-b1cd-7eea74a775bb None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.240432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8df566-d3ee-43ab-b1cd-7eea74a775bb None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.241283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e037b-706b-4c5a-8fdf-d54d33d82f06 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.241939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 298/613] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:41.249495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02041d-ee78-45db-ac87-f12c16f3a707 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.249837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02041d-ee78-45db-ac87-f12c16f3a707 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.250171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02041d-ee78-45db-ac87-f12c16f3a707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.276701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-6d02041d-ee78-45db-ac87-f12c16f3a707 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:41.278322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6d02041d-ee78-45db-ac87-f12c16f3a707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:41.284845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07d86c-8bd4-4ab0-842a-58fd0347055d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.285164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07d86c-8bd4-4ab0-842a-58fd0347055d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.285441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07d86c-8bd4-4ab0-842a-58fd0347055d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.285713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07d86c-8bd4-4ab0-842a-58fd0347055d None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.286458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07d86c-8bd4-4ab0-842a-58fd0347055d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:41.286789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07d86c-8bd4-4ab0-842a-58fd0347055d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.288301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07d86c-8bd4-4ab0-842a-58fd0347055d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.303079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 316/614] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:41.308721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07d86c-8bd4-4ab0-842a-58fd0347055d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.364678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.368459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 299/615] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:41.391627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02041d-ee78-45db-ac87-f12c16f3a707 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.434358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07d86c-8bd4-4ab0-842a-58fd0347055d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.481119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434ca36-1363-4076-af45-5fee217a716b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.481920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434ca36-1363-4076-af45-5fee217a716b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.482205 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434ca36-1363-4076-af45-5fee217a716b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.482499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434ca36-1363-4076-af45-5fee217a716b None admin] PATH_INFO: `/v3/users/nova` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.491762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434ca36-1363-4076-af45-5fee217a716b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.493321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434ca36-1363-4076-af45-5fee217a716b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-a434ca36-1363-4076-af45-5fee217a716b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:41.510877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:41.511884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:41.513261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 317/616] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:41.518745 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434ca36-1363-4076-af45-5fee217a716b None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.563978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02041d-ee78-45db-ac87-f12c16f3a707 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.579156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.630466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e5989-1ece-4aff-98d9-f6b97e353605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.631457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e5989-1ece-4aff-98d9-f6b97e353605 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.632065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e5989-1ece-4aff-98d9-f6b97e353605 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.632065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e5989-1ece-4aff-98d9-f6b97e353605 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.638220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434ca36-1363-4076-af45-5fee217a716b None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.650824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.661567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e5989-1ece-4aff-98d9-f6b97e353605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:41.675580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42835219-6294-4b2c-8b31-571930f0a296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-ab7e5989-1ece-4aff-98d9-f6b97e353605 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:41.676246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:41.677185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:41.678047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:41.684068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42835219-6294-4b2c-8b31-571930f0a296 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.684197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 300/617] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:41.684815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42835219-6294-4b2c-8b31-571930f0a296 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.685324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42835219-6294-4b2c-8b31-571930f0a296 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.685431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e5989-1ece-4aff-98d9-f6b97e353605 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.686699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42835219-6294-4b2c-8b31-571930f0a296 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:41.687224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42835219-6294-4b2c-8b31-571930f0a296 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.688911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42835219-6294-4b2c-8b31-571930f0a296 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.707833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 318/618] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:41.713513 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42835219-6294-4b2c-8b31-571930f0a296 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.820150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42835219-6294-4b2c-8b31-571930f0a296 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.861089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cedbf-594c-4bf7-9c0a-c71356598686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.862294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cedbf-594c-4bf7-9c0a-c71356598686 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.862833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cedbf-594c-4bf7-9c0a-c71356598686 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.863330 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cedbf-594c-4bf7-9c0a-c71356598686 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.874423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e5989-1ece-4aff-98d9-f6b97e353605 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.898914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.921360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5d098-d8d1-4d58-ac49-f13ce2cfd43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.922033 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5d098-d8d1-4d58-ac49-f13ce2cfd43d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.922311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5d098-d8d1-4d58-ac49-f13ce2cfd43d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.922623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5d098-d8d1-4d58-ac49-f13ce2cfd43d None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.923370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5d098-d8d1-4d58-ac49-f13ce2cfd43d None admin] RBAC: Adding query filter params (name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:41.923674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5d098-d8d1-4d58-ac49-f13ce2cfd43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.925147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5d098-d8d1-4d58-ac49-f13ce2cfd43d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.933666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 301/619] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:41.940858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5d098-d8d1-4d58-ac49-f13ce2cfd43d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.956714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cedbf-594c-4bf7-9c0a-c71356598686 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.958540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cedbf-594c-4bf7-9c0a-c71356598686 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.006160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.117659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5d098-d8d1-4d58-ac49-f13ce2cfd43d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.162321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 319/620] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:40:42.163523 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cedbf-594c-4bf7-9c0a-c71356598686 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.197360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e722e-23c6-433a-bbdc-293fcf814d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.200284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e722e-23c6-433a-bbdc-293fcf814d96 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.200780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e722e-23c6-433a-bbdc-293fcf814d96 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.201284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e722e-23c6-433a-bbdc-293fcf814d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.231563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e722e-23c6-433a-bbdc-293fcf814d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.233029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e722e-23c6-433a-bbdc-293fcf814d96 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.250955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-243e722e-23c6-433a-bbdc-293fcf814d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:42.271366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:42.272333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:42.273234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:42.273234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:42.273234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:42.273234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:42.274792 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 302/621] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:42.283253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e722e-23c6-433a-bbdc-293fcf814d96 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.318110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cedbf-594c-4bf7-9c0a-c71356598686 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.396939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.411686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe135f2-ee05-457b-8300-f4f65dd24da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.412681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe135f2-ee05-457b-8300-f4f65dd24da3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.413057 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe135f2-ee05-457b-8300-f4f65dd24da3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.413336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe135f2-ee05-457b-8300-f4f65dd24da3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.415242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e722e-23c6-433a-bbdc-293fcf814d96 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.428406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe135f2-ee05-457b-8300-f4f65dd24da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.430674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe135f2-ee05-457b-8300-f4f65dd24da3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-4fe135f2-ee05-457b-8300-f4f65dd24da3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:42.451487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:42.452465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:42.453382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:42.453382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:42.453382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:42.453382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:42.457337 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 320/622] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:42.458791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82042511-4045-4125-a8c7-7952f5f68b6a None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.459137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82042511-4045-4125-a8c7-7952f5f68b6a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.459339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82042511-4045-4125-a8c7-7952f5f68b6a None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.460915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 321/623] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:42.461214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db0008-d3c7-4bc8-8139-1e10f703215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.462088 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0008-d3c7-4bc8-8139-1e10f703215b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.462600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0008-d3c7-4bc8-8139-1e10f703215b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.463042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0008-d3c7-4bc8-8139-1e10f703215b None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.464470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0008-d3c7-4bc8-8139-1e10f703215b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:42.475149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0008-d3c7-4bc8-8139-1e10f703215b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.475291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a21cd-3400-4a20-81b2-ca34d1acb7ac None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.475376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0008-d3c7-4bc8-8139-1e10f703215b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.475450 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a21cd-3400-4a20-81b2-ca34d1acb7ac None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.475450 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a21cd-3400-4a20-81b2-ca34d1acb7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.482977 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 303/624] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:42.484273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db0008-d3c7-4bc8-8139-1e10f703215b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.490970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-fa8a21cd-3400-4a20-81b2-ca34d1acb7ac None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:42.492226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-fa8a21cd-3400-4a20-81b2-ca34d1acb7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:42.577908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 322/625] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:42.578895 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a21cd-3400-4a20-81b2-ca34d1acb7ac None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.617755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db0008-d3c7-4bc8-8139-1e10f703215b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.642114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.666706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef02b9-716b-49ea-b942-e94bf6e00ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.668393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef02b9-716b-49ea-b942-e94bf6e00ece None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.668893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef02b9-716b-49ea-b942-e94bf6e00ece None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.669405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef02b9-716b-49ea-b942-e94bf6e00ece None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.670329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef02b9-716b-49ea-b942-e94bf6e00ece None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:42.670873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef02b9-716b-49ea-b942-e94bf6e00ece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.672846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef02b9-716b-49ea-b942-e94bf6e00ece None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.689659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 304/626] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:42.690879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef02b9-716b-49ea-b942-e94bf6e00ece None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.702394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.704569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a21cd-3400-4a20-81b2-ca34d1acb7ac None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.744264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452b7b9-f25d-49f8-9e27-ccb564086127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.745793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452b7b9-f25d-49f8-9e27-ccb564086127 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.746704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452b7b9-f25d-49f8-9e27-ccb564086127 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.747316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452b7b9-f25d-49f8-9e27-ccb564086127 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.757585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452b7b9-f25d-49f8-9e27-ccb564086127 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.759539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452b7b9-f25d-49f8-9e27-ccb564086127 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-d452b7b9-f25d-49f8-9e27-ccb564086127 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:42.780629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:42.781628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:42.782644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:42.782644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:42.782644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:42.782644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:42.782644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:42.782644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:42.782644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:42.782644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:42.782644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:40:42.782644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:42.786607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 323/627] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:42.787614 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452b7b9-f25d-49f8-9e27-ccb564086127 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.813136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef02b9-716b-49ea-b942-e94bf6e00ece None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.850162 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6fd8d-8ff6-4f24-b591-4b2ef8084bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.850990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6fd8d-8ff6-4f24-b591-4b2ef8084bf9 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.851272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6fd8d-8ff6-4f24-b591-4b2ef8084bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.851548 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6fd8d-8ff6-4f24-b591-4b2ef8084bf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.862563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6fd8d-8ff6-4f24-b591-4b2ef8084bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-14f6fd8d-8ff6-4f24-b591-4b2ef8084bf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:42.870685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:42.871649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:42.872589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:42.876637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 305/628] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:42.877512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6fd8d-8ff6-4f24-b591-4b2ef8084bf9 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.928480 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452b7b9-f25d-49f8-9e27-ccb564086127 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.931694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.964893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.965202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcc774-8ce3-4b9e-a6c6-a39396eacc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.966031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc774-8ce3-4b9e-a6c6-a39396eacc22 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.966249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc774-8ce3-4b9e-a6c6-a39396eacc22 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.966556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcc774-8ce3-4b9e-a6c6-a39396eacc22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.973630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcc774-8ce3-4b9e-a6c6-a39396eacc22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.975125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcc774-8ce3-4b9e-a6c6-a39396eacc22 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-91dcc774-8ce3-4b9e-a6c6-a39396eacc22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:42.990630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:42.997435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:42.998351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:42.998351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:42.998351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:42.998351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:42.998351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:42.998351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:42.998351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:42.998351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:42.998351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 324/629] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:42.998351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcc774-8ce3-4b9e-a6c6-a39396eacc22 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.007466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6fd8d-8ff6-4f24-b591-4b2ef8084bf9 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.051833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dd9e3-4f3b-47ca-9850-1e8f5735e74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.052510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dd9e3-4f3b-47ca-9850-1e8f5735e74e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.052793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dd9e3-4f3b-47ca-9850-1e8f5735e74e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.053065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dd9e3-4f3b-47ca-9850-1e8f5735e74e None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.054370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dd9e3-4f3b-47ca-9850-1e8f5735e74e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:43.054712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dd9e3-4f3b-47ca-9850-1e8f5735e74e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.056521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dd9e3-4f3b-47ca-9850-1e8f5735e74e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.085353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 306/630] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:43.086589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dd9e3-4f3b-47ca-9850-1e8f5735e74e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.127167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcc774-8ce3-4b9e-a6c6-a39396eacc22 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.164760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f1f3f-1d33-46eb-9bf8-fec2311a4c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.165435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f1f3f-1d33-46eb-9bf8-fec2311a4c5a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.165720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f1f3f-1d33-46eb-9bf8-fec2311a4c5a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.165998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f1f3f-1d33-46eb-9bf8-fec2311a4c5a None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.166862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f1f3f-1d33-46eb-9bf8-fec2311a4c5a None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:43.167178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f1f3f-1d33-46eb-9bf8-fec2311a4c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.168811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f1f3f-1d33-46eb-9bf8-fec2311a4c5a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.183148 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 325/631] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:43.183961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f1f3f-1d33-46eb-9bf8-fec2311a4c5a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.195794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dd9e3-4f3b-47ca-9850-1e8f5735e74e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.214990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.221387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.232560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948a77b-4487-48e4-adff-cf74e378b41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.233541 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948a77b-4487-48e4-adff-cf74e378b41c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.233995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948a77b-4487-48e4-adff-cf74e378b41c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.234445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948a77b-4487-48e4-adff-cf74e378b41c None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.235465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948a77b-4487-48e4-adff-cf74e378b41c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:43.235985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948a77b-4487-48e4-adff-cf74e378b41c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.237684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948a77b-4487-48e4-adff-cf74e378b41c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.250805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 307/632] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:43.251894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948a77b-4487-48e4-adff-cf74e378b41c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.304293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f1f3f-1d33-46eb-9bf8-fec2311a4c5a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.343536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996910d-d034-4f93-80b9-aab4326c2a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.344337 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996910d-d034-4f93-80b9-aab4326c2a5f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.344715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996910d-d034-4f93-80b9-aab4326c2a5f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.344994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996910d-d034-4f93-80b9-aab4326c2a5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.356602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996910d-d034-4f93-80b9-aab4326c2a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.358066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996910d-d034-4f93-80b9-aab4326c2a5f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.358913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948a77b-4487-48e4-adff-cf74e378b41c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-3996910d-d034-4f93-80b9-aab4326c2a5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:43.377899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:43.378892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:43.379808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:43.379808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:43.379808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:43.379808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:43.379808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 326/633] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:43.380909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996910d-d034-4f93-80b9-aab4326c2a5f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.400116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b089b-2a18-4af6-b573-090648abd3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.401131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b089b-2a18-4af6-b573-090648abd3a2 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.401613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b089b-2a18-4af6-b573-090648abd3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.401886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b089b-2a18-4af6-b573-090648abd3a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.412914 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b089b-2a18-4af6-b573-090648abd3a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.414422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b089b-2a18-4af6-b573-090648abd3a2 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-751b089b-2a18-4af6-b573-090648abd3a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:43.431875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:43.433533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:43.435044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:43.435044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:43.435044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:43.435044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:43.435044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 308/634] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:43.435044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2ee79-13f4-4535-8d93-586ff14ef039 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.435393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2ee79-13f4-4535-8d93-586ff14ef039 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.435393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2ee79-13f4-4535-8d93-586ff14ef039 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.436811 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 309/635] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:43.439234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2ee79-13f4-4535-8d93-586ff14ef039 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.463462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.474434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.484517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996910d-d034-4f93-80b9-aab4326c2a5f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.517322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43c21a-86bf-4514-8598-1c9b9dc1b7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.517965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43c21a-86bf-4514-8598-1c9b9dc1b7ca None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.518242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43c21a-86bf-4514-8598-1c9b9dc1b7ca None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.518542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43c21a-86bf-4514-8598-1c9b9dc1b7ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.519373 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43c21a-86bf-4514-8598-1c9b9dc1b7ca None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=067b827625e0427294b93b027597a5f3) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:43.519677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43c21a-86bf-4514-8598-1c9b9dc1b7ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.521094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43c21a-86bf-4514-8598-1c9b9dc1b7ca None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.526819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 327/636] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/role_assignments?user.id=067b827625e0427294b93b027597a5f3&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:43.527995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a5ff4-36d4-4aaf-9ce8-d1d1a8f0a6ee None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.528293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a5ff4-36d4-4aaf-9ce8-d1d1a8f0a6ee None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.528574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a5ff4-36d4-4aaf-9ce8-d1d1a8f0a6ee None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.530064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 328/637] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:43.530912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a5ff4-36d4-4aaf-9ce8-d1d1a8f0a6ee None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.544717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2ee79-13f4-4535-8d93-586ff14ef039 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.584537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860455cc-a009-41e8-97fc-502642a6d4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.585229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860455cc-a009-41e8-97fc-502642a6d4a7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.585510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860455cc-a009-41e8-97fc-502642a6d4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.585784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860455cc-a009-41e8-97fc-502642a6d4a7 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.586548 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860455cc-a009-41e8-97fc-502642a6d4a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:43.586888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860455cc-a009-41e8-97fc-502642a6d4a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.588309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860455cc-a009-41e8-97fc-502642a6d4a7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.607026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 310/638] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:43.607026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1a40d-b6ea-4531-bfe9-add594dadc17 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.607026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1a40d-b6ea-4531-bfe9-add594dadc17 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.607026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1a40d-b6ea-4531-bfe9-add594dadc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.626222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-01f1a40d-b6ea-4531-bfe9-add594dadc17 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:43.627498 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-01f1a40d-b6ea-4531-bfe9-add594dadc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:43.667744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a5ff4-36d4-4aaf-9ce8-d1d1a8f0a6ee None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.708569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b639bb-4090-4446-aeac-ff158eaf649a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.709093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 311/639] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:43.709277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b639bb-4090-4446-aeac-ff158eaf649a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.709561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b639bb-4090-4446-aeac-ff158eaf649a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.710004 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b639bb-4090-4446-aeac-ff158eaf649a None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.710400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b57c8-f18e-43fb-b19c-50b5174b886d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.710723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b639bb-4090-4446-aeac-ff158eaf649a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:43.710818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b57c8-f18e-43fb-b19c-50b5174b886d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.711030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b57c8-f18e-43fb-b19c-50b5174b886d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.711110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b639bb-4090-4446-aeac-ff158eaf649a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.712412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b639bb-4090-4446-aeac-ff158eaf649a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.724323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 329/640] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:43.726767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b639bb-4090-4446-aeac-ff158eaf649a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.730399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.731272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0b3b57c8-f18e-43fb-b19c-50b5174b886d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:43.732475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0b3b57c8-f18e-43fb-b19c-50b5174b886d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:43.757877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.813238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 312/641] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:43.814083 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b57c8-f18e-43fb-b19c-50b5174b886d None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.840934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b639bb-4090-4446-aeac-ff158eaf649a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.875743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aa425-caf8-453f-abdb-c4be902e3070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.876544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aa425-caf8-453f-abdb-c4be902e3070 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.876821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aa425-caf8-453f-abdb-c4be902e3070 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.877092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aa425-caf8-453f-abdb-c4be902e3070 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.883951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aa425-caf8-453f-abdb-c4be902e3070 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.885450 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aa425-caf8-453f-abdb-c4be902e3070 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-d26aa425-caf8-453f-abdb-c4be902e3070 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:43.900005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:43.901107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:43.902628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:43.902628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:43.902628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:43.902628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:43.902628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:43.902628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:43.902628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:43.902628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:43.902628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 330/642] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:43.902628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aa425-caf8-453f-abdb-c4be902e3070 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.934988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b57c8-f18e-43fb-b19c-50b5174b886d None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.968776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1f098-7230-45a6-8f0e-e2ad99f2ad64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.969507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1f098-7230-45a6-8f0e-e2ad99f2ad64 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.969645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.970056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1f098-7230-45a6-8f0e-e2ad99f2ad64 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.970056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1f098-7230-45a6-8f0e-e2ad99f2ad64 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.977256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1f098-7230-45a6-8f0e-e2ad99f2ad64 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.978738 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1f098-7230-45a6-8f0e-e2ad99f2ad64 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-56f1f098-7230-45a6-8f0e-e2ad99f2ad64 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:43.993697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:44.003552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 313/643] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:44.004743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1f098-7230-45a6-8f0e-e2ad99f2ad64 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:44.015323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:44.019643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aa425-caf8-453f-abdb-c4be902e3070 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:44.059048 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf439-086f-4f21-a1c6-66200167e369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:44.059852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf439-086f-4f21-a1c6-66200167e369 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.060133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf439-086f-4f21-a1c6-66200167e369 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.060409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf439-086f-4f21-a1c6-66200167e369 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.071037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2cf439-086f-4f21-a1c6-66200167e369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-1b2cf439-086f-4f21-a1c6-66200167e369 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:44.079021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:44.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:44.080849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 331/644] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:44.081651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf439-086f-4f21-a1c6-66200167e369 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:44.107825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1f098-7230-45a6-8f0e-e2ad99f2ad64 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:44.146036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb900e-0898-4d49-b2d3-f65ff24318f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:44.146917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb900e-0898-4d49-b2d3-f65ff24318f1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.147198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb900e-0898-4d49-b2d3-f65ff24318f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.147521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb900e-0898-4d49-b2d3-f65ff24318f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.158989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb900e-0898-4d49-b2d3-f65ff24318f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-3ebb900e-0898-4d49-b2d3-f65ff24318f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:44.167098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:44.168051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:44.168915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 314/645] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:44.169622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb900e-0898-4d49-b2d3-f65ff24318f1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:44.194360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf439-086f-4f21-a1c6-66200167e369 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:44.217160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:44.228733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba28e5-738e-4cb8-b900-b8c754068600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:44.229360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba28e5-738e-4cb8-b900-b8c754068600 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.229646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba28e5-738e-4cb8-b900-b8c754068600 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.229927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba28e5-738e-4cb8-b900-b8c754068600 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.230873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba28e5-738e-4cb8-b900-b8c754068600 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:44.231196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba28e5-738e-4cb8-b900-b8c754068600 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:44.232656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba28e5-738e-4cb8-b900-b8c754068600 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:44.244990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 332/646] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:44.245803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba28e5-738e-4cb8-b900-b8c754068600 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:44.256496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:44.281576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb900e-0898-4d49-b2d3-f65ff24318f1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:44.321632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc99ef1-dca4-4ac3-a4ee-11788dec1e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:44.322498 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc99ef1-dca4-4ac3-a4ee-11788dec1e38 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.322796 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc99ef1-dca4-4ac3-a4ee-11788dec1e38 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.324148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc99ef1-dca4-4ac3-a4ee-11788dec1e38 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.325094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc99ef1-dca4-4ac3-a4ee-11788dec1e38 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:44.325411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc99ef1-dca4-4ac3-a4ee-11788dec1e38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:44.327125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc99ef1-dca4-4ac3-a4ee-11788dec1e38 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:44.345586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 315/647] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 05:40:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:44.346421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc99ef1-dca4-4ac3-a4ee-11788dec1e38 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:44.360840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba28e5-738e-4cb8-b900-b8c754068600 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:44.398439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2ee64-f093-453b-8b30-5b80d96feb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:44.399187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2ee64-f093-453b-8b30-5b80d96feb4c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.399469 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2ee64-f093-453b-8b30-5b80d96feb4c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.399744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2ee64-f093-453b-8b30-5b80d96feb4c None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.400497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2ee64-f093-453b-8b30-5b80d96feb4c None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:44.400807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2ee64-f093-453b-8b30-5b80d96feb4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:44.402848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2ee64-f093-453b-8b30-5b80d96feb4c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:44.412755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 333/648] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:44.413755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2ee64-f093-453b-8b30-5b80d96feb4c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:44.461602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:44.475297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc99ef1-dca4-4ac3-a4ee-11788dec1e38 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:44.508390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c224335-01dd-48fd-9aae-0fa7760afc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:44.508992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c224335-01dd-48fd-9aae-0fa7760afc20 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.509273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c224335-01dd-48fd-9aae-0fa7760afc20 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.509555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c224335-01dd-48fd-9aae-0fa7760afc20 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.510294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c224335-01dd-48fd-9aae-0fa7760afc20 None admin] RBAC: Adding query filter params (name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:44.511160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c224335-01dd-48fd-9aae-0fa7760afc20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:44.512529 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c224335-01dd-48fd-9aae-0fa7760afc20 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:44.514103 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:44.520118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2ee64-f093-453b-8b30-5b80d96feb4c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:44.520738 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 316/649] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:44.521517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c224335-01dd-48fd-9aae-0fa7760afc20 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:44.559345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279833e-cb2d-44ad-95f9-18e37ec1f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:44.560237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279833e-cb2d-44ad-95f9-18e37ec1f003 None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.560521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279833e-cb2d-44ad-95f9-18e37ec1f003 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.560794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279833e-cb2d-44ad-95f9-18e37ec1f003 None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/5c359fc0168141fb8da3f953884fd2e5/roles/9cc15ca9e61f425cacb9c1a9f52073dd` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.577289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279833e-cb2d-44ad-95f9-18e37ec1f003 None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=5c359fc0168141fb8da3f953884fd2e5, role_id=9cc15ca9e61f425cacb9c1a9f52073dd)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:44.578941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279833e-cb2d-44ad-95f9-18e37ec1f003 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:44.619429 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c224335-01dd-48fd-9aae-0fa7760afc20 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:44.652910 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8c21e-f2a7-4d5a-938c-284d11629762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:44.653617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8c21e-f2a7-4d5a-938c-284d11629762 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.653906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8c21e-f2a7-4d5a-938c-284d11629762 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.654184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8c21e-f2a7-4d5a-938c-284d11629762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.720320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8c21e-f2a7-4d5a-938c-284d11629762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:44.721795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8c21e-f2a7-4d5a-938c-284d11629762 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:44.759737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 334/650] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:44 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/5c359fc0168141fb8da3f953884fd2e5/roles/9cc15ca9e61f425cacb9c1a9f52073dd => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:44.761409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279833e-cb2d-44ad-95f9-18e37ec1f003 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:44.764542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:44.862630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 317/651] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:44.863950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8c21e-f2a7-4d5a-938c-284d11629762 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:44.902040 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:44.915809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279833e-cb2d-44ad-95f9-18e37ec1f003 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:44.957335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8d21e-ef6d-40f3-accc-c2b768aa4f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:44.958136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8d21e-ef6d-40f3-accc-c2b768aa4f84 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.958417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8d21e-ef6d-40f3-accc-c2b768aa4f84 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.958720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8d21e-ef6d-40f3-accc-c2b768aa4f84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.971184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8d21e-ef6d-40f3-accc-c2b768aa4f84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:44.972913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8d21e-ef6d-40f3-accc-c2b768aa4f84 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:44.987229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-31a8d21e-ef6d-40f3-accc-c2b768aa4f84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:44.995607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:45.001582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:45.002584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:45.002584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:45.002584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:45.002584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:45.002837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8c21e-f2a7-4d5a-938c-284d11629762 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.002924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 335/652] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:45.002924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8d21e-ef6d-40f3-accc-c2b768aa4f84 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.041309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afb683-359d-4702-b6be-d934cca02ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.042675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afb683-359d-4702-b6be-d934cca02ea7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.043023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afb683-359d-4702-b6be-d934cca02ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.043365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afb683-359d-4702-b6be-d934cca02ea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.055478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afb683-359d-4702-b6be-d934cca02ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.058079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afb683-359d-4702-b6be-d934cca02ea7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-74afb683-359d-4702-b6be-d934cca02ea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:45.080674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:45.081626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:45.082831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:45.082831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:45.082831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:45.082831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:45.086540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 318/653] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:45.087557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afb683-359d-4702-b6be-d934cca02ea7 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.122246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8d21e-ef6d-40f3-accc-c2b768aa4f84 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.156779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08377c7e-1731-4318-923b-0ad0494c8a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.157946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08377c7e-1731-4318-923b-0ad0494c8a80 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.158446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08377c7e-1731-4318-923b-0ad0494c8a80 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.158950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08377c7e-1731-4318-923b-0ad0494c8a80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.169978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08377c7e-1731-4318-923b-0ad0494c8a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.171765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08377c7e-1731-4318-923b-0ad0494c8a80 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-08377c7e-1731-4318-923b-0ad0494c8a80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:45.190693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:45.191650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:45.192544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:45.192544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:45.192544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:45.192544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:45.194805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 336/654] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:45.196142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08377c7e-1731-4318-923b-0ad0494c8a80 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.210005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afb683-359d-4702-b6be-d934cca02ea7 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.214845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.248317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7e53b-4831-4b53-a459-088e84d6032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.248991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7e53b-4831-4b53-a459-088e84d6032f None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.249270 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7e53b-4831-4b53-a459-088e84d6032f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.249544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7e53b-4831-4b53-a459-088e84d6032f None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.250426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7e53b-4831-4b53-a459-088e84d6032f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:45.250841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7e53b-4831-4b53-a459-088e84d6032f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.252628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7e53b-4831-4b53-a459-088e84d6032f None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.270566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 319/655] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:45.271671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7e53b-4831-4b53-a459-088e84d6032f None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.287480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.324284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08377c7e-1731-4318-923b-0ad0494c8a80 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.369064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4293-ff75-444b-ba50-b3910a2b300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.369923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4293-ff75-444b-ba50-b3910a2b300f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.370629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4293-ff75-444b-ba50-b3910a2b300f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.370910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4293-ff75-444b-ba50-b3910a2b300f None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.371640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c4293-ff75-444b-ba50-b3910a2b300f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:45.372211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c4293-ff75-444b-ba50-b3910a2b300f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.373742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c4293-ff75-444b-ba50-b3910a2b300f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.386690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7e53b-4831-4b53-a459-088e84d6032f None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.388002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 337/656] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:45.389329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4293-ff75-444b-ba50-b3910a2b300f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.437739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb16f6-5578-449c-a4e1-46346fd6cd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.438652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb16f6-5578-449c-a4e1-46346fd6cd81 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.438977 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb16f6-5578-449c-a4e1-46346fd6cd81 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.439258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb16f6-5578-449c-a4e1-46346fd6cd81 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.440003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb16f6-5578-449c-a4e1-46346fd6cd81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:45.440300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb16f6-5578-449c-a4e1-46346fd6cd81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.441664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb16f6-5578-449c-a4e1-46346fd6cd81 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.458169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 320/657] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:45.458169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb16f6-5578-449c-a4e1-46346fd6cd81 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.471603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.517485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4293-ff75-444b-ba50-b3910a2b300f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.545724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.555600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4488c-3c80-4061-b61e-348c5392d181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.556321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4488c-3c80-4061-b61e-348c5392d181 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.556690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4488c-3c80-4061-b61e-348c5392d181 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.557063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4488c-3c80-4061-b61e-348c5392d181 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.565469 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4488c-3c80-4061-b61e-348c5392d181 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.567141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4488c-3c80-4061-b61e-348c5392d181 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-34d4488c-3c80-4061-b61e-348c5392d181 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:45.582151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:45.583384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:45.584556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:45.584556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:45.584556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:45.584556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:45.584556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:45.584556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:45.584556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:45.584556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:45.584556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:40:45.584556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:45.591082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 338/658] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:45.591082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4488c-3c80-4061-b61e-348c5392d181 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.592639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb16f6-5578-449c-a4e1-46346fd6cd81 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.630368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ce70-ec0e-40fd-89c7-21c8a6f3bc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.631530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ce70-ec0e-40fd-89c7-21c8a6f3bc01 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.631952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ce70-ec0e-40fd-89c7-21c8a6f3bc01 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.632401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ce70-ec0e-40fd-89c7-21c8a6f3bc01 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.641494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4ce70-ec0e-40fd-89c7-21c8a6f3bc01 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.643174 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4ce70-ec0e-40fd-89c7-21c8a6f3bc01 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-fde4ce70-ec0e-40fd-89c7-21c8a6f3bc01 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:45.659357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:45.660508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:45.661547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:45.661547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:45.661547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:45.661547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:45.661547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:45.661547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:45.661547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:45.661547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:45.661547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:40:45.661547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:45.663702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 321/659] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:45.664667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ce70-ec0e-40fd-89c7-21c8a6f3bc01 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.691623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4488c-3c80-4061-b61e-348c5392d181 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.708343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.725841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd326e27-f17a-4f33-9f7b-9ce8e07d1ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.726669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd326e27-f17a-4f33-9f7b-9ce8e07d1ad8 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.726954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd326e27-f17a-4f33-9f7b-9ce8e07d1ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.727229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd326e27-f17a-4f33-9f7b-9ce8e07d1ad8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.733735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd326e27-f17a-4f33-9f7b-9ce8e07d1ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.735228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd326e27-f17a-4f33-9f7b-9ce8e07d1ad8 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-bd326e27-f17a-4f33-9f7b-9ce8e07d1ad8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:45.749184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:45.750287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:45.751375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:45.751375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:45.751375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:45.751375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:45.751375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:45.751375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:45.751375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:45.751375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:45.751375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 339/660] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:45.751897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfa2cd-b86d-4910-9c90-36ea7bfbc09c None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.752162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfa2cd-b86d-4910-9c90-36ea7bfbc09c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.752465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfa2cd-b86d-4910-9c90-36ea7bfbc09c None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.753845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 340/661] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:45.754381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfa2cd-b86d-4910-9c90-36ea7bfbc09c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.785807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ce70-ec0e-40fd-89c7-21c8a6f3bc01 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.813094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.821872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841dcb88-4040-4173-b237-8415cf4796d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.824677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dcb88-4040-4173-b237-8415cf4796d6 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.824677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dcb88-4040-4173-b237-8415cf4796d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.824677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dcb88-4040-4173-b237-8415cf4796d6 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.824677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841dcb88-4040-4173-b237-8415cf4796d6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:45.825003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841dcb88-4040-4173-b237-8415cf4796d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.826575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841dcb88-4040-4173-b237-8415cf4796d6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.848136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 322/662] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:45.848136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841dcb88-4040-4173-b237-8415cf4796d6 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.868696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfa2cd-b86d-4910-9c90-36ea7bfbc09c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.917668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ddf97-b9f7-49a5-9c71-6aff8dd1398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.918374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ddf97-b9f7-49a5-9c71-6aff8dd1398f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.918673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ddf97-b9f7-49a5-9c71-6aff8dd1398f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.918949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ddf97-b9f7-49a5-9c71-6aff8dd1398f None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.919792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ddf97-b9f7-49a5-9c71-6aff8dd1398f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:45.920099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ddf97-b9f7-49a5-9c71-6aff8dd1398f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.921544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ddf97-b9f7-49a5-9c71-6aff8dd1398f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.944941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 341/663] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:45.944941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978304-0655-4742-92ff-5111514ddc22 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.944941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978304-0655-4742-92ff-5111514ddc22 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.944941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978304-0655-4742-92ff-5111514ddc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.959220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4d978304-0655-4742-92ff-5111514ddc22 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:45.960421 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4d978304-0655-4742-92ff-5111514ddc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:45.964456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841dcb88-4040-4173-b237-8415cf4796d6 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.988898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.001670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468dd8e-ecab-49fc-ab76-eef6a8e989a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.002333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468dd8e-ecab-49fc-ab76-eef6a8e989a9 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.002997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468dd8e-ecab-49fc-ab76-eef6a8e989a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.003279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468dd8e-ecab-49fc-ab76-eef6a8e989a9 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.004064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468dd8e-ecab-49fc-ab76-eef6a8e989a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:46.007405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468dd8e-ecab-49fc-ab76-eef6a8e989a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.007405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468dd8e-ecab-49fc-ab76-eef6a8e989a9 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.024044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 323/664] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:46.025127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468dd8e-ecab-49fc-ab76-eef6a8e989a9 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.035244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 342/665] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:46.036242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d978304-0655-4742-92ff-5111514ddc22 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.075772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.133097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468dd8e-ecab-49fc-ab76-eef6a8e989a9 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.145301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d978304-0655-4742-92ff-5111514ddc22 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.172200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57a76b-5e45-4ac3-be7c-537cd4fc79fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.173188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57a76b-5e45-4ac3-be7c-537cd4fc79fc None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.173697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57a76b-5e45-4ac3-be7c-537cd4fc79fc None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.174189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57a76b-5e45-4ac3-be7c-537cd4fc79fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.180017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb4b1-2604-4cc8-8218-8edc17463378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.180695 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb4b1-2604-4cc8-8218-8edc17463378 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.180977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb4b1-2604-4cc8-8218-8edc17463378 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.181253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb4b1-2604-4cc8-8218-8edc17463378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.216517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.245851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57a76b-5e45-4ac3-be7c-537cd4fc79fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.247335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57a76b-5e45-4ac3-be7c-537cd4fc79fc None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.251780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bb4b1-2604-4cc8-8218-8edc17463378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.253672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bb4b1-2604-4cc8-8218-8edc17463378 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.295613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.351640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 324/666] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:46.354569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 343/667] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:46.356892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57a76b-5e45-4ac3-be7c-537cd4fc79fc None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.357227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb4b1-2604-4cc8-8218-8edc17463378 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.422726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.457777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb4b1-2604-4cc8-8218-8edc17463378 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.473611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57a76b-5e45-4ac3-be7c-537cd4fc79fc None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.492247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f0fae-87a1-475f-9519-eb2bdf82239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.493059 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f0fae-87a1-475f-9519-eb2bdf82239d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.493340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f0fae-87a1-475f-9519-eb2bdf82239d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.493616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f0fae-87a1-475f-9519-eb2bdf82239d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.507099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f0fae-87a1-475f-9519-eb2bdf82239d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:46.515359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832137f2-049d-4831-918e-73c99652af61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-c76f0fae-87a1-475f-9519-eb2bdf82239d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:46.515732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:46.516664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:46.517535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:46.518197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832137f2-049d-4831-918e-73c99652af61 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.518197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832137f2-049d-4831-918e-73c99652af61 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.518197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832137f2-049d-4831-918e-73c99652af61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.518197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832137f2-049d-4831-918e-73c99652af61 None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=6e12361da5f849ee97607ca68ee6ef10) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:46.518197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832137f2-049d-4831-918e-73c99652af61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.519114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832137f2-049d-4831-918e-73c99652af61 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.520316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 344/668] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:46.521335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f0fae-87a1-475f-9519-eb2bdf82239d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.529646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 325/669] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/role_assignments?user.id=6e12361da5f849ee97607ca68ee6ef10&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:46.536101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832137f2-049d-4831-918e-73c99652af61 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.553446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.665600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f0fae-87a1-475f-9519-eb2bdf82239d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.680863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832137f2-049d-4831-918e-73c99652af61 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.705439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.717669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa1ca8-4e50-4e71-a327-ff3ca1198f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.720809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa1ca8-4e50-4e71-a327-ff3ca1198f8e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.720809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa1ca8-4e50-4e71-a327-ff3ca1198f8e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.720809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa1ca8-4e50-4e71-a327-ff3ca1198f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.723151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cb416-4854-4b83-bec4-648b028e1c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.723943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cb416-4854-4b83-bec4-648b028e1c6e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.724221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cb416-4854-4b83-bec4-648b028e1c6e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.724492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cb416-4854-4b83-bec4-648b028e1c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.731901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa1ca8-4e50-4e71-a327-ff3ca1198f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.733441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa1ca8-4e50-4e71-a327-ff3ca1198f8e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.737982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cb416-4854-4b83-bec4-648b028e1c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.739475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cb416-4854-4b83-bec4-648b028e1c6e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-2efa1ca8-4e50-4e71-a327-ff3ca1198f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:46.756159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:46.757110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:46.758015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:46.758015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:46.758015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:46.758015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:46.758015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 345/670] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:46.758762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa1ca8-4e50-4e71-a327-ff3ca1198f8e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-989cb416-4854-4b83-bec4-648b028e1c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:46.758881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:46.759712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:46.760648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:46.760648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:46.760648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:46.760648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:46.760648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 326/671] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:46.766781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cb416-4854-4b83-bec4-648b028e1c6e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.849029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.901648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa1ca8-4e50-4e71-a327-ff3ca1198f8e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.904032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cb416-4854-4b83-bec4-648b028e1c6e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.941889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bfee9-8d89-44e5-a993-3008b88c2be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.942248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad985085-64b1-4a79-8166-d48131456d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.942493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bfee9-8d89-44e5-a993-3008b88c2be2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.942785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bfee9-8d89-44e5-a993-3008b88c2be2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.942979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad985085-64b1-4a79-8166-d48131456d61 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.943060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bfee9-8d89-44e5-a993-3008b88c2be2 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.943222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad985085-64b1-4a79-8166-d48131456d61 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.943470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad985085-64b1-4a79-8166-d48131456d61 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.943789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bfee9-8d89-44e5-a993-3008b88c2be2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:46.944091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bfee9-8d89-44e5-a993-3008b88c2be2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.944201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad985085-64b1-4a79-8166-d48131456d61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:46.944475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad985085-64b1-4a79-8166-d48131456d61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.945455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bfee9-8d89-44e5-a993-3008b88c2be2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.945815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad985085-64b1-4a79-8166-d48131456d61 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.954642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 346/672] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:46.955518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bfee9-8d89-44e5-a993-3008b88c2be2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.966715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 327/673] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:46.966715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5324325-4d75-4856-ac3d-36f7ccda38e7 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.966715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5324325-4d75-4856-ac3d-36f7ccda38e7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.966715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5324325-4d75-4856-ac3d-36f7ccda38e7 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.966715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 328/674] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:46.967142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5324325-4d75-4856-ac3d-36f7ccda38e7 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.002192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:47.062288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bfee9-8d89-44e5-a993-3008b88c2be2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.089547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5324325-4d75-4856-ac3d-36f7ccda38e7 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.096940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4356c2-b81e-4d18-9d71-518ec10b1030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.097609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4356c2-b81e-4d18-9d71-518ec10b1030 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.097886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4356c2-b81e-4d18-9d71-518ec10b1030 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.098158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4356c2-b81e-4d18-9d71-518ec10b1030 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.098953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4356c2-b81e-4d18-9d71-518ec10b1030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:47.099266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4356c2-b81e-4d18-9d71-518ec10b1030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.100664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4356c2-b81e-4d18-9d71-518ec10b1030 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.117582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 347/675] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:47.117582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4356c2-b81e-4d18-9d71-518ec10b1030 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.118723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:47.131893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ee274-a4e6-4d38-8a9b-d37bc7667284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.138580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ee274-a4e6-4d38-8a9b-d37bc7667284 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.138580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ee274-a4e6-4d38-8a9b-d37bc7667284 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.138580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ee274-a4e6-4d38-8a9b-d37bc7667284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.145671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ee274-a4e6-4d38-8a9b-d37bc7667284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.147176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ee274-a4e6-4d38-8a9b-d37bc7667284 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-184ee274-a4e6-4d38-8a9b-d37bc7667284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:47.167878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:47.168838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:47.169737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:47.169737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:47.169737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:47.169737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:47.169737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 329/676] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:47.170566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964e93e-e666-436a-b20e-e648285c3f7d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.170566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964e93e-e666-436a-b20e-e648285c3f7d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.170852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964e93e-e666-436a-b20e-e648285c3f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.186956 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-d964e93e-e666-436a-b20e-e648285c3f7d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:47.188143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d964e93e-e666-436a-b20e-e648285c3f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:47.229807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:47.232813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4356c2-b81e-4d18-9d71-518ec10b1030 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.262572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 330/677] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:47.264265 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964e93e-e666-436a-b20e-e648285c3f7d None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.273983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2d726-9801-4c93-91c1-1720db137a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.274975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2d726-9801-4c93-91c1-1720db137a60 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.275535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2d726-9801-4c93-91c1-1720db137a60 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.275957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2d726-9801-4c93-91c1-1720db137a60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.283691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2d726-9801-4c93-91c1-1720db137a60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.285534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2d726-9801-4c93-91c1-1720db137a60 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-50f2d726-9801-4c93-91c1-1720db137a60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:47.299444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:47.300516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:47.301624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:47.301624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:47.301624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:47.301624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:47.301624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:47.301624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:47.301624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:47.301624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:47.303403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 348/678] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:47.304397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2d726-9801-4c93-91c1-1720db137a60 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.385964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964e93e-e666-436a-b20e-e648285c3f7d None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.389656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:47.410616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2d726-9801-4c93-91c1-1720db137a60 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.420924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55c5a4-ffe9-4ae9-a8d4-9dc43e214e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.421720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55c5a4-ffe9-4ae9-a8d4-9dc43e214e0b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.421997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55c5a4-ffe9-4ae9-a8d4-9dc43e214e0b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.422259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55c5a4-ffe9-4ae9-a8d4-9dc43e214e0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.429196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a55c5a4-ffe9-4ae9-a8d4-9dc43e214e0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.431422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a55c5a4-ffe9-4ae9-a8d4-9dc43e214e0b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-9a55c5a4-ffe9-4ae9-a8d4-9dc43e214e0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:47.445517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:47.446524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:47.447451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:47.447451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:47.447451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:47.447451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:47.447451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:47.447451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:47.447451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:47.447451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:47.447451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 331/679] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:47.449108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55c5a4-ffe9-4ae9-a8d4-9dc43e214e0b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.450992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc2cda-dfc7-4b74-b7a4-895fdb81ba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.451655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc2cda-dfc7-4b74-b7a4-895fdb81ba68 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.452221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc2cda-dfc7-4b74-b7a4-895fdb81ba68 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.452502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc2cda-dfc7-4b74-b7a4-895fdb81ba68 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.453900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc2cda-dfc7-4b74-b7a4-895fdb81ba68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:47.453900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc2cda-dfc7-4b74-b7a4-895fdb81ba68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.454901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc2cda-dfc7-4b74-b7a4-895fdb81ba68 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.472200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 349/680] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:47.472200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc2cda-dfc7-4b74-b7a4-895fdb81ba68 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.493958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:47.568402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55c5a4-ffe9-4ae9-a8d4-9dc43e214e0b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.572849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc2cda-dfc7-4b74-b7a4-895fdb81ba68 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.606212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3baf94-7709-4965-a0e9-a961428e3b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.607046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3baf94-7709-4965-a0e9-a961428e3b08 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.607327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3baf94-7709-4965-a0e9-a961428e3b08 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.607599 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3baf94-7709-4965-a0e9-a961428e3b08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.608323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec8d0d-9a2a-4d3b-9164-05d562cf9364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.608978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec8d0d-9a2a-4d3b-9164-05d562cf9364 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.609369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec8d0d-9a2a-4d3b-9164-05d562cf9364 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.609563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec8d0d-9a2a-4d3b-9164-05d562cf9364 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.610300 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec8d0d-9a2a-4d3b-9164-05d562cf9364 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:47.610689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec8d0d-9a2a-4d3b-9164-05d562cf9364 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.612221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec8d0d-9a2a-4d3b-9164-05d562cf9364 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.617751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3baf94-7709-4965-a0e9-a961428e3b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:47.628403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 350/681] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:47.628403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec8d0d-9a2a-4d3b-9164-05d562cf9364 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-ac3baf94-7709-4965-a0e9-a961428e3b08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:47.628630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:47.629500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:47.630435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:47.631740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 332/682] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:47.632266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3baf94-7709-4965-a0e9-a961428e3b08 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.659788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:47.719103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:47.739419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec8d0d-9a2a-4d3b-9164-05d562cf9364 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.739588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3baf94-7709-4965-a0e9-a961428e3b08 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.775857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbd097-f968-4d26-96fb-bc777ed379bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.776706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd541e4-785e-43dd-b5c6-f42d0eeec627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.777104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbd097-f968-4d26-96fb-bc777ed379bc None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.777326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd541e4-785e-43dd-b5c6-f42d0eeec627 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.777426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbd097-f968-4d26-96fb-bc777ed379bc None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.777603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd541e4-785e-43dd-b5c6-f42d0eeec627 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.777700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbd097-f968-4d26-96fb-bc777ed379bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.777861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd541e4-785e-43dd-b5c6-f42d0eeec627 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.778671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd541e4-785e-43dd-b5c6-f42d0eeec627 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:47.778980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd541e4-785e-43dd-b5c6-f42d0eeec627 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.780794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd541e4-785e-43dd-b5c6-f42d0eeec627 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.785429 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbd097-f968-4d26-96fb-bc777ed379bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.786901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbd097-f968-4d26-96fb-bc777ed379bc None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.798001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 351/683] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:47.798001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd541e4-785e-43dd-b5c6-f42d0eeec627 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-52cbd097-f968-4d26-96fb-bc777ed379bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:47.806915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:47.808026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:47.808982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 333/684] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:47.809605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbd097-f968-4d26-96fb-bc777ed379bc None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.902351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:47.914716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd541e4-785e-43dd-b5c6-f42d0eeec627 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.917714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbd097-f968-4d26-96fb-bc777ed379bc None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.949016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f978-5365-485f-89e3-b5ab2362d6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.949670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f978-5365-485f-89e3-b5ab2362d6ca None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.949948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f978-5365-485f-89e3-b5ab2362d6ca None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.950228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f978-5365-485f-89e3-b5ab2362d6ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.951087 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1f978-5365-485f-89e3-b5ab2362d6ca None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=6640e4c799514429a2fe7850369af4df) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:47.951454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1f978-5365-485f-89e3-b5ab2362d6ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.952821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1f978-5365-485f-89e3-b5ab2362d6ca None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.953289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc56e1e-95f1-4be7-b43a-6cdeee271108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.953950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc56e1e-95f1-4be7-b43a-6cdeee271108 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.954230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc56e1e-95f1-4be7-b43a-6cdeee271108 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.954547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc56e1e-95f1-4be7-b43a-6cdeee271108 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.955277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc56e1e-95f1-4be7-b43a-6cdeee271108 None admin] RBAC: Adding query filter params (name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:47.955589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc56e1e-95f1-4be7-b43a-6cdeee271108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.956919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc56e1e-95f1-4be7-b43a-6cdeee271108 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.958863 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 352/685] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/role_assignments?user.id=6640e4c799514429a2fe7850369af4df&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:47.959791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f978-5365-485f-89e3-b5ab2362d6ca None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.971551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:47.974626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 334/686] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:47.975575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc56e1e-95f1-4be7-b43a-6cdeee271108 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.078950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f978-5365-485f-89e3-b5ab2362d6ca None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.092274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc56e1e-95f1-4be7-b43a-6cdeee271108 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.115376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa5f0b-9aab-4b06-84c8-d1dabe984bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.116047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa5f0b-9aab-4b06-84c8-d1dabe984bc8 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.116322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa5f0b-9aab-4b06-84c8-d1dabe984bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.116594 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa5f0b-9aab-4b06-84c8-d1dabe984bc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.117443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa5f0b-9aab-4b06-84c8-d1dabe984bc8 None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=ecd61347f8874b89aada19772ec56179) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:48.117752 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa5f0b-9aab-4b06-84c8-d1dabe984bc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.119376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa5f0b-9aab-4b06-84c8-d1dabe984bc8 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.130893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 353/687] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/role_assignments?user.id=ecd61347f8874b89aada19772ec56179&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:48.131835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa5f0b-9aab-4b06-84c8-d1dabe984bc8 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.132401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67538e-37e9-465f-8c19-3ebd5589ac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.133344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67538e-37e9-465f-8c19-3ebd5589ac2c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.133825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67538e-37e9-465f-8c19-3ebd5589ac2c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.134298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67538e-37e9-465f-8c19-3ebd5589ac2c None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.135402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67538e-37e9-465f-8c19-3ebd5589ac2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:48.136487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67538e-37e9-465f-8c19-3ebd5589ac2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.138169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67538e-37e9-465f-8c19-3ebd5589ac2c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.159059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 335/688] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 05:40:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:48.164952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67538e-37e9-465f-8c19-3ebd5589ac2c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.171214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.240248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.283347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67538e-37e9-465f-8c19-3ebd5589ac2c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.289018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa5f0b-9aab-4b06-84c8-d1dabe984bc8 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.327776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168e045-cf84-4783-a1ff-b18a153192a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.328825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168e045-cf84-4783-a1ff-b18a153192a2 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.329361 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168e045-cf84-4783-a1ff-b18a153192a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.329900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168e045-cf84-4783-a1ff-b18a153192a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.330274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39834476-37b7-4a40-bc0e-43f2801bd0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.331217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39834476-37b7-4a40-bc0e-43f2801bd0af None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.331517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39834476-37b7-4a40-bc0e-43f2801bd0af None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.331786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39834476-37b7-4a40-bc0e-43f2801bd0af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.347448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39834476-37b7-4a40-bc0e-43f2801bd0af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.348934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39834476-37b7-4a40-bc0e-43f2801bd0af None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-39834476-37b7-4a40-bc0e-43f2801bd0af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:48.366810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:48.367800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:48.368721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:48.368721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:48.368721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:48.368721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:48.373246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 354/689] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:48.374553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39834476-37b7-4a40-bc0e-43f2801bd0af None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.419945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168e045-cf84-4783-a1ff-b18a153192a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.422122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168e045-cf84-4783-a1ff-b18a153192a2 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.426700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.488474 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39834476-37b7-4a40-bc0e-43f2801bd0af None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.515151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.530831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bccba7-d7a9-45c5-a28d-7a4cce37a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.531520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bccba7-d7a9-45c5-a28d-7a4cce37a66e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.531800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bccba7-d7a9-45c5-a28d-7a4cce37a66e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.532080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bccba7-d7a9-45c5-a28d-7a4cce37a66e None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.532821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bccba7-d7a9-45c5-a28d-7a4cce37a66e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:48.533124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bccba7-d7a9-45c5-a28d-7a4cce37a66e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.534666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bccba7-d7a9-45c5-a28d-7a4cce37a66e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.551476 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 355/690] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:48.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bccba7-d7a9-45c5-a28d-7a4cce37a66e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.561753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 336/691] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:48.566993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168e045-cf84-4783-a1ff-b18a153192a2 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.665461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bccba7-d7a9-45c5-a28d-7a4cce37a66e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.674215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168e045-cf84-4783-a1ff-b18a153192a2 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.702874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.711770 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f6564-06fe-422a-abf7-4362e03c754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.715883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f6564-06fe-422a-abf7-4362e03c754d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.715883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f6564-06fe-422a-abf7-4362e03c754d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.715883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f6564-06fe-422a-abf7-4362e03c754d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.726864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f6564-06fe-422a-abf7-4362e03c754d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.728616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f6564-06fe-422a-abf7-4362e03c754d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-255f6564-06fe-422a-abf7-4362e03c754d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:48.748620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:48.749662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:48.750595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:48.750595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:48.750595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:48.750595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:48.753222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 337/692] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:48.757656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81acaf-0eba-41a8-b662-f2d3bc4a58c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.758764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f6564-06fe-422a-abf7-4362e03c754d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.761159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81acaf-0eba-41a8-b662-f2d3bc4a58c3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.763592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81acaf-0eba-41a8-b662-f2d3bc4a58c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.765128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81acaf-0eba-41a8-b662-f2d3bc4a58c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.783630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81acaf-0eba-41a8-b662-f2d3bc4a58c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.785103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81acaf-0eba-41a8-b662-f2d3bc4a58c3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-9a81acaf-0eba-41a8-b662-f2d3bc4a58c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:48.810541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:48.811558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:48.813151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 356/693] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:48.813752 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618228f-e7df-42ed-a0ec-71ad98b968ee None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.813752 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618228f-e7df-42ed-a0ec-71ad98b968ee None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.814122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618228f-e7df-42ed-a0ec-71ad98b968ee None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.819239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 357/694] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:48.820161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618228f-e7df-42ed-a0ec-71ad98b968ee None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.869362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.898407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f6564-06fe-422a-abf7-4362e03c754d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.933342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618228f-e7df-42ed-a0ec-71ad98b968ee None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.955014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08754a-5372-4dd9-b4ae-5fc7053fae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.956359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08754a-5372-4dd9-b4ae-5fc7053fae63 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.956808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08754a-5372-4dd9-b4ae-5fc7053fae63 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.957224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08754a-5372-4dd9-b4ae-5fc7053fae63 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.958094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08754a-5372-4dd9-b4ae-5fc7053fae63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:48.958565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08754a-5372-4dd9-b4ae-5fc7053fae63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.960619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08754a-5372-4dd9-b4ae-5fc7053fae63 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.974959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a89ad-9f26-4ad1-bd6d-3ac3b7c271ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.975831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a89ad-9f26-4ad1-bd6d-3ac3b7c271ba None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.976228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a89ad-9f26-4ad1-bd6d-3ac3b7c271ba None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.976650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a89ad-9f26-4ad1-bd6d-3ac3b7c271ba None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.977592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a89ad-9f26-4ad1-bd6d-3ac3b7c271ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:48.978143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a89ad-9f26-4ad1-bd6d-3ac3b7c271ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.978328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 338/695] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:48.979852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a89ad-9f26-4ad1-bd6d-3ac3b7c271ba None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.980023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ada5b-bf84-42e7-9d85-6315ba2de5cb None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.980441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ada5b-bf84-42e7-9d85-6315ba2de5cb None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.980871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ada5b-bf84-42e7-9d85-6315ba2de5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.000291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 358/696] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:49.001323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a89ad-9f26-4ad1-bd6d-3ac3b7c271ba None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.001929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-640ada5b-bf84-42e7-9d85-6315ba2de5cb None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:49.003163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-640ada5b-bf84-42e7-9d85-6315ba2de5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:49.017573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.090769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 339/697] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:49.092047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ada5b-bf84-42e7-9d85-6315ba2de5cb None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.118705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.124710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a89ad-9f26-4ad1-bd6d-3ac3b7c271ba None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.168573 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1280569-71a9-480f-b095-2db0e060308a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.169801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1280569-71a9-480f-b095-2db0e060308a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.170555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1280569-71a9-480f-b095-2db0e060308a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.170983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1280569-71a9-480f-b095-2db0e060308a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.179828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1280569-71a9-480f-b095-2db0e060308a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.181617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1280569-71a9-480f-b095-2db0e060308a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-d1280569-71a9-480f-b095-2db0e060308a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:49.199434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:49.200461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:49.201358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:49.201358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:49.201358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:49.201358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:49.201358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:49.201358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:49.201358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:49.201358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:49.202039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 359/698] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:49.202935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1280569-71a9-480f-b095-2db0e060308a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.223528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ada5b-bf84-42e7-9d85-6315ba2de5cb None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.269207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb11b23-5535-4a41-a522-f957d567d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.270108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb11b23-5535-4a41-a522-f957d567d74e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.270703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb11b23-5535-4a41-a522-f957d567d74e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.271138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb11b23-5535-4a41-a522-f957d567d74e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.290215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.315734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1280569-71a9-480f-b095-2db0e060308a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.352585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511074ce-7a17-43be-b106-e6a6c6162271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.353923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511074ce-7a17-43be-b106-e6a6c6162271 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.354432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511074ce-7a17-43be-b106-e6a6c6162271 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.354955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511074ce-7a17-43be-b106-e6a6c6162271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.356420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb11b23-5535-4a41-a522-f957d567d74e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.358044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb11b23-5535-4a41-a522-f957d567d74e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.367749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511074ce-7a17-43be-b106-e6a6c6162271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.369447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511074ce-7a17-43be-b106-e6a6c6162271 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-511074ce-7a17-43be-b106-e6a6c6162271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:49.388366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:49.389549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:49.390512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:49.390512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:49.390512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:49.390512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:49.397199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 360/699] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:49.398278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511074ce-7a17-43be-b106-e6a6c6162271 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.411111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.483015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 340/700] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:49.483872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb11b23-5535-4a41-a522-f957d567d74e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.506917 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511074ce-7a17-43be-b106-e6a6c6162271 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.534941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.543588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bf46e-f3c9-464d-bb36-3a246cb41758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.544375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bf46e-f3c9-464d-bb36-3a246cb41758 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.544759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bf46e-f3c9-464d-bb36-3a246cb41758 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.545138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bf46e-f3c9-464d-bb36-3a246cb41758 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.546048 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bf46e-f3c9-464d-bb36-3a246cb41758 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:49.546457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bf46e-f3c9-464d-bb36-3a246cb41758 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.548026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bf46e-f3c9-464d-bb36-3a246cb41758 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.561289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 361/701] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:49.562321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bf46e-f3c9-464d-bb36-3a246cb41758 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.589634 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb11b23-5535-4a41-a522-f957d567d74e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.625233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afc7b7-6fb5-4713-a39d-eb0cf823761d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.626002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afc7b7-6fb5-4713-a39d-eb0cf823761d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.626290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afc7b7-6fb5-4713-a39d-eb0cf823761d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.626596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afc7b7-6fb5-4713-a39d-eb0cf823761d None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.627316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afc7b7-6fb5-4713-a39d-eb0cf823761d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:49.627624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afc7b7-6fb5-4713-a39d-eb0cf823761d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.629531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afc7b7-6fb5-4713-a39d-eb0cf823761d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.634240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.644266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 341/702] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:49.645548 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afc7b7-6fb5-4713-a39d-eb0cf823761d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.674243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bf46e-f3c9-464d-bb36-3a246cb41758 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.709933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e3882-b4a6-4d78-a351-a1eff423b214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.710677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e3882-b4a6-4d78-a351-a1eff423b214 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.710962 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e3882-b4a6-4d78-a351-a1eff423b214 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.711239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e3882-b4a6-4d78-a351-a1eff423b214 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.723385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e3882-b4a6-4d78-a351-a1eff423b214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.724872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e3882-b4a6-4d78-a351-a1eff423b214 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-170e3882-b4a6-4d78-a351-a1eff423b214 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:49.742873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:49.743806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:49.745432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:49.745432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:49.745432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:49.745432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:49.745432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 362/703] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:49.745432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e3882-b4a6-4d78-a351-a1eff423b214 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.757310 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afc7b7-6fb5-4713-a39d-eb0cf823761d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.770827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.795076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba665f51-1a93-4acc-b4d7-b5cc2214716e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.804135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba665f51-1a93-4acc-b4d7-b5cc2214716e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.804135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba665f51-1a93-4acc-b4d7-b5cc2214716e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.804135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba665f51-1a93-4acc-b4d7-b5cc2214716e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.804135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba665f51-1a93-4acc-b4d7-b5cc2214716e None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=067b827625e0427294b93b027597a5f3) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:49.804135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba665f51-1a93-4acc-b4d7-b5cc2214716e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.804135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba665f51-1a93-4acc-b4d7-b5cc2214716e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.806737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 342/704] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/role_assignments?user.id=067b827625e0427294b93b027597a5f3&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=7a833c45603a4830b888e58e1232712a => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:49.808436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba665f51-1a93-4acc-b4d7-b5cc2214716e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.851881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e3882-b4a6-4d78-a351-a1eff423b214 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.878235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.897683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e361e-329e-41cb-933b-93b0a4e91835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.898922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e361e-329e-41cb-933b-93b0a4e91835 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.899211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e361e-329e-41cb-933b-93b0a4e91835 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.899485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e361e-329e-41cb-933b-93b0a4e91835 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.900284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e361e-329e-41cb-933b-93b0a4e91835 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.902003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e361e-329e-41cb-933b-93b0a4e91835 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.971172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba665f51-1a93-4acc-b4d7-b5cc2214716e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.016064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 363/705] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:49 2022] POST /identity/v3/users => generated 254 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:50.016960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77e82c-94c5-42fb-9257-3624ab2cd73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.017664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77e82c-94c5-42fb-9257-3624ab2cd73f None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.017943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77e82c-94c5-42fb-9257-3624ab2cd73f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.018215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77e82c-94c5-42fb-9257-3624ab2cd73f None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.018974 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77e82c-94c5-42fb-9257-3624ab2cd73f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:50.019277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77e82c-94c5-42fb-9257-3624ab2cd73f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:50.020665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77e82c-94c5-42fb-9257-3624ab2cd73f None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:50.044926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 343/706] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:50.050294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e361e-329e-41cb-933b-93b0a4e91835 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.205870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e361e-329e-41cb-933b-93b0a4e91835 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.241899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ef5a6-f9f5-44cd-a286-ab59247b32f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.242806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ef5a6-f9f5-44cd-a286-ab59247b32f9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.243086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ef5a6-f9f5-44cd-a286-ab59247b32f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.243359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ef5a6-f9f5-44cd-a286-ab59247b32f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.251094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ef5a6-f9f5-44cd-a286-ab59247b32f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:50.252567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ef5a6-f9f5-44cd-a286-ab59247b32f9 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5d0ef5a6-f9f5-44cd-a286-ab59247b32f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:50.268655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:50.269579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:50.270916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:50.270916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:50.270916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:50.270916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:50.270916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:50.270916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:50.270916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:50.270916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:50.270916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 364/707] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:50.278056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77e82c-94c5-42fb-9257-3624ab2cd73f None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.312112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:50.388493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77e82c-94c5-42fb-9257-3624ab2cd73f None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.426960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04a403-e495-4012-ae78-e1101167115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.428126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04a403-e495-4012-ae78-e1101167115d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.428611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04a403-e495-4012-ae78-e1101167115d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.429086 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04a403-e495-4012-ae78-e1101167115d None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.430071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04a403-e495-4012-ae78-e1101167115d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:50.430600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04a403-e495-4012-ae78-e1101167115d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:50.432324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04a403-e495-4012-ae78-e1101167115d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:50.441630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e982a28-193c-41d6-a8ec-c4d07c7bc62f None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.441931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e982a28-193c-41d6-a8ec-c4d07c7bc62f None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.442188 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e982a28-193c-41d6-a8ec-c4d07c7bc62f None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.444631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 365/708] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:50.448929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 344/709] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:50.449093 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9aaf1-61b7-466d-a014-c66fe760a8a7 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.449187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9aaf1-61b7-466d-a014-c66fe760a8a7 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.449446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9aaf1-61b7-466d-a014-c66fe760a8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.455354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04a403-e495-4012-ae78-e1101167115d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.480106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1da9aaf1-61b7-466d-a014-c66fe760a8a7 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:50.481340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1da9aaf1-61b7-466d-a014-c66fe760a8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:50.516900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:50.560345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:50.569480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04a403-e495-4012-ae78-e1101167115d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.577116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 366/710] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:50.606683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9aaf1-61b7-466d-a014-c66fe760a8a7 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.613110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931f8aa-8b46-4337-947e-d336c889dda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.614172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931f8aa-8b46-4337-947e-d336c889dda2 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.614506 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931f8aa-8b46-4337-947e-d336c889dda2 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.618869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931f8aa-8b46-4337-947e-d336c889dda2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.621397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931f8aa-8b46-4337-947e-d336c889dda2 None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=5c359fc0168141fb8da3f953884fd2e5) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:50.621721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931f8aa-8b46-4337-947e-d336c889dda2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:50.624419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931f8aa-8b46-4337-947e-d336c889dda2 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:50.633205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 345/711] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/role_assignments?user.id=5c359fc0168141fb8da3f953884fd2e5&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:50.720461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7979005-9655-4047-9141-90d3816862c7 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.720755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7979005-9655-4047-9141-90d3816862c7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.721014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7979005-9655-4047-9141-90d3816862c7 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.722424 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 346/712] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:50.731957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f728f-10e8-422c-9bb5-0471a04749ef None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.732268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f728f-10e8-422c-9bb5-0471a04749ef None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.732539 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f728f-10e8-422c-9bb5-0471a04749ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.759744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9aaf1-61b7-466d-a014-c66fe760a8a7 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.766291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-535f728f-10e8-422c-9bb5-0471a04749ef None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:50.767536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-535f728f-10e8-422c-9bb5-0471a04749ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:50.827811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b08f57-2983-4a73-929b-8a4a8d0f4570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.828567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b08f57-2983-4a73-929b-8a4a8d0f4570 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.828845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b08f57-2983-4a73-929b-8a4a8d0f4570 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.829116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b08f57-2983-4a73-929b-8a4a8d0f4570 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.829782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b08f57-2983-4a73-929b-8a4a8d0f4570 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:50.831197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b08f57-2983-4a73-929b-8a4a8d0f4570 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-58b08f57-2983-4a73-929b-8a4a8d0f4570 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:50.849453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:50.850584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:40:50.851541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:50.852377 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 367/713] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:50.858336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b08f57-2983-4a73-929b-8a4a8d0f4570 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.893339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 347/714] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:50.905451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:50.919196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f728f-10e8-422c-9bb5-0471a04749ef None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.992457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b08f57-2983-4a73-929b-8a4a8d0f4570 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:51.016221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:51.035371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f728f-10e8-422c-9bb5-0471a04749ef None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:51.036502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1b97f-c6ed-4df3-8f3b-a867b45e4faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:51.037338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1b97f-c6ed-4df3-8f3b-a867b45e4faf None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:51.038414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1b97f-c6ed-4df3-8f3b-a867b45e4faf None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:51.038722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1b97f-c6ed-4df3-8f3b-a867b45e4faf None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:51.039446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1b97f-c6ed-4df3-8f3b-a867b45e4faf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:51.039758 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1b97f-c6ed-4df3-8f3b-a867b45e4faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:51.041159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1b97f-c6ed-4df3-8f3b-a867b45e4faf None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:51.047191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 368/715] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:51.052321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1b97f-c6ed-4df3-8f3b-a867b45e4faf None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:51.081229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cfe85-587c-4356-903d-6a44806b31c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:51.082121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cfe85-587c-4356-903d-6a44806b31c4 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:51.082414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cfe85-587c-4356-903d-6a44806b31c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:51.083036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cfe85-587c-4356-903d-6a44806b31c4 None admin] PATH_INFO: `/v3/services/network` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:51.083982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cfe85-587c-4356-903d-6a44806b31c4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:51.085568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cfe85-587c-4356-903d-6a44806b31c4 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-ff1cfe85-587c-4356-903d-6a44806b31c4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:51.103242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:51.104332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:51.105201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 348/716] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:51.111238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cfe85-587c-4356-903d-6a44806b31c4 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:51.162560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:51.170126 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1b97f-c6ed-4df3-8f3b-a867b45e4faf None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:51.205823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5434cf-8bf8-4a38-8315-c089dff7dcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:51.206501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5434cf-8bf8-4a38-8315-c089dff7dcb1 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:51.207103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5434cf-8bf8-4a38-8315-c089dff7dcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:51.207103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5434cf-8bf8-4a38-8315-c089dff7dcb1 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:51.207777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5434cf-8bf8-4a38-8315-c089dff7dcb1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:51.208083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5434cf-8bf8-4a38-8315-c089dff7dcb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:51.209586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5434cf-8bf8-4a38-8315-c089dff7dcb1 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:51.216440 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 369/717] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 05:40:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:51.236788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cfe85-587c-4356-903d-6a44806b31c4 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:51.275371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994733e4-9cce-4990-adca-ea5f629046cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:51.276618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994733e4-9cce-4990-adca-ea5f629046cb None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:51.276902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994733e4-9cce-4990-adca-ea5f629046cb None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:51.277173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994733e4-9cce-4990-adca-ea5f629046cb None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:51.277897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994733e4-9cce-4990-adca-ea5f629046cb None admin] RBAC: Adding query filter params (name=network) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:51.278200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994733e4-9cce-4990-adca-ea5f629046cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:51.279697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994733e4-9cce-4990-adca-ea5f629046cb None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:51.285345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 349/718] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:40:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:51.295752 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5434cf-8bf8-4a38-8315-c089dff7dcb1 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:51.453270 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5434cf-8bf8-4a38-8315-c089dff7dcb1 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:51.487256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24beb6ad-1330-4abe-83aa-5417540a67c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:51.487929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beb6ad-1330-4abe-83aa-5417540a67c8 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:51.488210 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beb6ad-1330-4abe-83aa-5417540a67c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:51.488488 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beb6ad-1330-4abe-83aa-5417540a67c8 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:51.489200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24beb6ad-1330-4abe-83aa-5417540a67c8 None admin] RBAC: Adding query filter params (type=network) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:51.489507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24beb6ad-1330-4abe-83aa-5417540a67c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:51.490926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24beb6ad-1330-4abe-83aa-5417540a67c8 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:51.496346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 370/719] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:40:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:52.344979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2a85f-cd13-4ce3-bfed-89e612bedd2b None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.345660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2a85f-cd13-4ce3-bfed-89e612bedd2b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.345660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2a85f-cd13-4ce3-bfed-89e612bedd2b None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.347225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 350/720] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:52.352239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cb76f-7670-49fb-9e68-a469ae934f19 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.353052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cb76f-7670-49fb-9e68-a469ae934f19 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.353578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cb76f-7670-49fb-9e68-a469ae934f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.379639 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d75cb76f-7670-49fb-9e68-a469ae934f19 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:52.381173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d75cb76f-7670-49fb-9e68-a469ae934f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:52.405807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:52.483195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 371/721] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:52.518492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2a85f-cd13-4ce3-bfed-89e612bedd2b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:52.544819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:52.676443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2a85f-cd13-4ce3-bfed-89e612bedd2b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:52.697483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea8a8d-bc63-4121-a738-041fabfd7027 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.698351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea8a8d-bc63-4121-a738-041fabfd7027 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.698596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea8a8d-bc63-4121-a738-041fabfd7027 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.700231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 372/722] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:52.704992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87acdca-d03f-4aa9-ba87-b6f8fed27ef0 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.705278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87acdca-d03f-4aa9-ba87-b6f8fed27ef0 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.705553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87acdca-d03f-4aa9-ba87-b6f8fed27ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.722648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f87acdca-d03f-4aa9-ba87-b6f8fed27ef0 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:52.723827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f87acdca-d03f-4aa9-ba87-b6f8fed27ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:52.741125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcbd17-476b-4747-bdea-b0be59a30862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:52.741931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcbd17-476b-4747-bdea-b0be59a30862 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.742207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcbd17-476b-4747-bdea-b0be59a30862 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.742501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcbd17-476b-4747-bdea-b0be59a30862 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.762332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcbd17-476b-4747-bdea-b0be59a30862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-1cbcbd17-476b-4747-bdea-b0be59a30862 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:52.770256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:52.771230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:52.772085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:52.781923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 351/723] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:52.790146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcbd17-476b-4747-bdea-b0be59a30862 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:52.848805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 373/724] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:52.881110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:52.912776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87acdca-d03f-4aa9-ba87-b6f8fed27ef0 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:52.921076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcbd17-476b-4747-bdea-b0be59a30862 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:52.958031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcec8d-35be-4f53-8c99-d9b925256b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:52.958658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcec8d-35be-4f53-8c99-d9b925256b60 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.958937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcec8d-35be-4f53-8c99-d9b925256b60 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.959209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcec8d-35be-4f53-8c99-d9b925256b60 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.959953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcec8d-35be-4f53-8c99-d9b925256b60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:52.960352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcec8d-35be-4f53-8c99-d9b925256b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:52.962408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcec8d-35be-4f53-8c99-d9b925256b60 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:52.975781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 352/725] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:52.983415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcec8d-35be-4f53-8c99-d9b925256b60 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.095986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.125528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcec8d-35be-4f53-8c99-d9b925256b60 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.129327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87acdca-d03f-4aa9-ba87-b6f8fed27ef0 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.187554 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0742e0-e1f8-4076-9e44-ccbf0d9e457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.188528 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0742e0-e1f8-4076-9e44-ccbf0d9e457f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.188912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0742e0-e1f8-4076-9e44-ccbf0d9e457f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.189369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0742e0-e1f8-4076-9e44-ccbf0d9e457f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.195748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cb751-b7ad-4e4b-96ed-d6027fb11075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.196758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cb751-b7ad-4e4b-96ed-d6027fb11075 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.197172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cb751-b7ad-4e4b-96ed-d6027fb11075 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.197587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cb751-b7ad-4e4b-96ed-d6027fb11075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.200796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0742e0-e1f8-4076-9e44-ccbf0d9e457f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:53.202426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-1e0742e0-e1f8-4076-9e44-ccbf0d9e457f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:53.210062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:53.211025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:53.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:53.214651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 374/726] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:53.219378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cb751-b7ad-4e4b-96ed-d6027fb11075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.220433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0742e0-e1f8-4076-9e44-ccbf0d9e457f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.222101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cb751-b7ad-4e4b-96ed-d6027fb11075 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-2c6cb751-b7ad-4e4b-96ed-d6027fb11075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:53.259329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:53.260383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:53.261302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:53.261302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:53.261302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:53.261302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:53.261302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 353/727] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:53.268899 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cb751-b7ad-4e4b-96ed-d6027fb11075 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.343293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0742e0-e1f8-4076-9e44-ccbf0d9e457f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.381097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0869c9a-e412-4bef-83e9-f32ae011098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.383318 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0869c9a-e412-4bef-83e9-f32ae011098c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.383790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0869c9a-e412-4bef-83e9-f32ae011098c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.384221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0869c9a-e412-4bef-83e9-f32ae011098c None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.385132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0869c9a-e412-4bef-83e9-f32ae011098c None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:53.385634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0869c9a-e412-4bef-83e9-f32ae011098c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.387208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0869c9a-e412-4bef-83e9-f32ae011098c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.401138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 375/728] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:53.402795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a60bbe-a711-47ad-a19b-fc65f34def0b None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.403258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a60bbe-a711-47ad-a19b-fc65f34def0b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.403702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a60bbe-a711-47ad-a19b-fc65f34def0b None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.406023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 376/729] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:53.406991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a60bbe-a711-47ad-a19b-fc65f34def0b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.420024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.424209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cb751-b7ad-4e4b-96ed-d6027fb11075 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.463597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bca13-9be2-4d2a-8a97-b1c90e79a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.464363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bca13-9be2-4d2a-8a97-b1c90e79a308 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.464737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bca13-9be2-4d2a-8a97-b1c90e79a308 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.465146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bca13-9be2-4d2a-8a97-b1c90e79a308 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.465967 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bca13-9be2-4d2a-8a97-b1c90e79a308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:53.466425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bca13-9be2-4d2a-8a97-b1c90e79a308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.467961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bca13-9be2-4d2a-8a97-b1c90e79a308 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.481666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 354/730] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:53.486636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd1c4-378c-44d1-8907-40fed08af7c3 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.486636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd1c4-378c-44d1-8907-40fed08af7c3 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.486636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd1c4-378c-44d1-8907-40fed08af7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.504518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-312cd1c4-378c-44d1-8907-40fed08af7c3 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:53.505718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-312cd1c4-378c-44d1-8907-40fed08af7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:53.537636 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a60bbe-a711-47ad-a19b-fc65f34def0b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.552728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.576977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa59d93-ae36-4f46-92d6-e52ebe6bb08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.578146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa59d93-ae36-4f46-92d6-e52ebe6bb08d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.578716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa59d93-ae36-4f46-92d6-e52ebe6bb08d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.579272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa59d93-ae36-4f46-92d6-e52ebe6bb08d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.586748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 355/731] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:53.587676 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cd1c4-378c-44d1-8907-40fed08af7c3 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.592395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa59d93-ae36-4f46-92d6-e52ebe6bb08d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.594401 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa59d93-ae36-4f46-92d6-e52ebe6bb08d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-faa59d93-ae36-4f46-92d6-e52ebe6bb08d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:53.615109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:53.616049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:53.616971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:53.616971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:53.616971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:53.616971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:53.619332 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 377/732] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:53.624329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa59d93-ae36-4f46-92d6-e52ebe6bb08d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.707743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cd1c4-378c-44d1-8907-40fed08af7c3 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.746410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445e3a2-ae6c-4339-ace1-c28bb940d81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.747181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e3a2-ae6c-4339-ace1-c28bb940d81a None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.747466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e3a2-ae6c-4339-ace1-c28bb940d81a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.747740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e3a2-ae6c-4339-ace1-c28bb940d81a None admin] PATH_INFO: `/v3/users/nova` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.756104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445e3a2-ae6c-4339-ace1-c28bb940d81a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.757584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445e3a2-ae6c-4339-ace1-c28bb940d81a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-8445e3a2-ae6c-4339-ace1-c28bb940d81a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:53.773697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:53.775011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:53.776597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:53.776597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:53.776597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:53.776597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:53.776597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:53.776597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:53.776597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:53.776597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:53.776597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:40:53.776597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:53.778812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 356/733] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:53.780063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445e3a2-ae6c-4339-ace1-c28bb940d81a None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.812874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.819604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.855591 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa59d93-ae36-4f46-92d6-e52ebe6bb08d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.890322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef452c87-7f25-4622-afa1-93e97eec6b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.891032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef452c87-7f25-4622-afa1-93e97eec6b43 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.891307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef452c87-7f25-4622-afa1-93e97eec6b43 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.891578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef452c87-7f25-4622-afa1-93e97eec6b43 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.892296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef452c87-7f25-4622-afa1-93e97eec6b43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:53.892597 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef452c87-7f25-4622-afa1-93e97eec6b43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.893988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef452c87-7f25-4622-afa1-93e97eec6b43 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.896677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445e3a2-ae6c-4339-ace1-c28bb940d81a None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.912392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 378/734] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:53.913770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345d729-6dde-46ab-8029-88ea43856de6 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.914124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345d729-6dde-46ab-8029-88ea43856de6 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.914462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345d729-6dde-46ab-8029-88ea43856de6 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.915958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 379/735] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:53.919235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345d729-6dde-46ab-8029-88ea43856de6 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.940697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bc90f-a9d0-442a-b953-18fc36a65001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.941479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bc90f-a9d0-442a-b953-18fc36a65001 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.941762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bc90f-a9d0-442a-b953-18fc36a65001 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.942404 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bc90f-a9d0-442a-b953-18fc36a65001 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.953193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bc90f-a9d0-442a-b953-18fc36a65001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-4d8bc90f-a9d0-442a-b953-18fc36a65001 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:53.961013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:53.961972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:53.962883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 357/736] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:53.964193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bc90f-a9d0-442a-b953-18fc36a65001 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.034805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345d729-6dde-46ab-8029-88ea43856de6 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.059707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.066958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.073891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bc90f-a9d0-442a-b953-18fc36a65001 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.077202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61712667-fb86-4bc2-9b8f-893c5e2dfb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.078062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61712667-fb86-4bc2-9b8f-893c5e2dfb88 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.078518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61712667-fb86-4bc2-9b8f-893c5e2dfb88 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.078955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61712667-fb86-4bc2-9b8f-893c5e2dfb88 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.079929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61712667-fb86-4bc2-9b8f-893c5e2dfb88 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:54.080381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61712667-fb86-4bc2-9b8f-893c5e2dfb88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.082012 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61712667-fb86-4bc2-9b8f-893c5e2dfb88 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.101389 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 380/737] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:54.102757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d1f85-9863-4016-8460-7e9109781c22 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.103045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d1f85-9863-4016-8460-7e9109781c22 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.103320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d1f85-9863-4016-8460-7e9109781c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.117788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c953e61-7ecc-4807-9886-ed3cad836d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.118633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c953e61-7ecc-4807-9886-ed3cad836d5c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.118918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c953e61-7ecc-4807-9886-ed3cad836d5c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.119193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c953e61-7ecc-4807-9886-ed3cad836d5c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.121970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-998d1f85-9863-4016-8460-7e9109781c22 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:54.123390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-998d1f85-9863-4016-8460-7e9109781c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:54.127365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c953e61-7ecc-4807-9886-ed3cad836d5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.128849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c953e61-7ecc-4807-9886-ed3cad836d5c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-8c953e61-7ecc-4807-9886-ed3cad836d5c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:54.144101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:54.145076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 358/738] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:54.146039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c953e61-7ecc-4807-9886-ed3cad836d5c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.199628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 381/739] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:54.200575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d1f85-9863-4016-8460-7e9109781c22 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.273244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c953e61-7ecc-4807-9886-ed3cad836d5c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.306717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d1f85-9863-4016-8460-7e9109781c22 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.309699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89223f21-2b79-46fd-ae3e-9333d6e46cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.310371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89223f21-2b79-46fd-ae3e-9333d6e46cfa None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.310685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89223f21-2b79-46fd-ae3e-9333d6e46cfa None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.310962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89223f21-2b79-46fd-ae3e-9333d6e46cfa None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.311705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89223f21-2b79-46fd-ae3e-9333d6e46cfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:54.312013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89223f21-2b79-46fd-ae3e-9333d6e46cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.313383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89223f21-2b79-46fd-ae3e-9333d6e46cfa None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.314304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.324052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 359/740] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:54.326902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89223f21-2b79-46fd-ae3e-9333d6e46cfa None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.346781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.347630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cf1ce-8450-45a3-a369-21f41eb64b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.348436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cf1ce-8450-45a3-a369-21f41eb64b8d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.348728 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cf1ce-8450-45a3-a369-21f41eb64b8d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.348988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cf1ce-8450-45a3-a369-21f41eb64b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.359072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cf1ce-8450-45a3-a369-21f41eb64b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.360516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cf1ce-8450-45a3-a369-21f41eb64b8d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-4b2cf1ce-8450-45a3-a369-21f41eb64b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:54.378200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:54.379461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:54.380645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:54.380645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:54.380645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:54.380645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:54.383679 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 382/741] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:54.383679 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cf1ce-8450-45a3-a369-21f41eb64b8d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.427695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89223f21-2b79-46fd-ae3e-9333d6e46cfa None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.461232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca9635-5570-40d0-b00d-6f23e3259ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.461898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca9635-5570-40d0-b00d-6f23e3259ec8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.462180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca9635-5570-40d0-b00d-6f23e3259ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.462455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca9635-5570-40d0-b00d-6f23e3259ec8 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.463301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca9635-5570-40d0-b00d-6f23e3259ec8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:54.463669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca9635-5570-40d0-b00d-6f23e3259ec8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.465791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca9635-5570-40d0-b00d-6f23e3259ec8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.483793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cf1ce-8450-45a3-a369-21f41eb64b8d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.490449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 360/742] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:54.491125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca9635-5570-40d0-b00d-6f23e3259ec8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.521765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669121e-477e-4d9a-b1b7-431e10c60f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.522431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669121e-477e-4d9a-b1b7-431e10c60f05 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.522760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669121e-477e-4d9a-b1b7-431e10c60f05 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.523040 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669121e-477e-4d9a-b1b7-431e10c60f05 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.523764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669121e-477e-4d9a-b1b7-431e10c60f05 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.525161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669121e-477e-4d9a-b1b7-431e10c60f05 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.597399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.603009 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 383/743] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:40:54 2022] POST /identity/v3/services => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:54.604358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74a23e-b3cc-465f-b747-1c4879caefc3 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.604649 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74a23e-b3cc-465f-b747-1c4879caefc3 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.604921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74a23e-b3cc-465f-b747-1c4879caefc3 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.610517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 384/744] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:54.610517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74a23e-b3cc-465f-b747-1c4879caefc3 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.623185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca9635-5570-40d0-b00d-6f23e3259ec8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.639289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.677042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7534e-992b-4030-91bd-6c17e3c196b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.677908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7534e-992b-4030-91bd-6c17e3c196b0 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.678191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7534e-992b-4030-91bd-6c17e3c196b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.678506 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7534e-992b-4030-91bd-6c17e3c196b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.694659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7534e-992b-4030-91bd-6c17e3c196b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.696651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7534e-992b-4030-91bd-6c17e3c196b0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-9bd7534e-992b-4030-91bd-6c17e3c196b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:54.718117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:54.719328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:54.720431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:54.720431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:54.720431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:54.720431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:54.720431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 361/745] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:54.720823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7534e-992b-4030-91bd-6c17e3c196b0 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.734931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74a23e-b3cc-465f-b747-1c4879caefc3 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.776361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ab11f-f689-4371-ac42-0215dfd5765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.777132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab11f-f689-4371-ac42-0215dfd5765f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.777478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab11f-f689-4371-ac42-0215dfd5765f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.777759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab11f-f689-4371-ac42-0215dfd5765f None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.779227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ab11f-f689-4371-ac42-0215dfd5765f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:54.780497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ab11f-f689-4371-ac42-0215dfd5765f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.781810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ab11f-f689-4371-ac42-0215dfd5765f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.794365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 385/746] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:54.795918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214d917-08f0-4391-b957-c4308ab5f9bd None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.796133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214d917-08f0-4391-b957-c4308ab5f9bd None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.796413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214d917-08f0-4391-b957-c4308ab5f9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.816331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e214d917-08f0-4391-b957-c4308ab5f9bd None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:54.817954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e214d917-08f0-4391-b957-c4308ab5f9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:54.836674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7534e-992b-4030-91bd-6c17e3c196b0 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.862808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.876225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00c11d-38f9-430b-92a5-2346fd4007bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.877446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00c11d-38f9-430b-92a5-2346fd4007bb None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.877728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00c11d-38f9-430b-92a5-2346fd4007bb None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.878141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00c11d-38f9-430b-92a5-2346fd4007bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.891208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 386/747] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:54.892388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214d917-08f0-4391-b957-c4308ab5f9bd None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.903096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.963919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00c11d-38f9-430b-92a5-2346fd4007bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.965479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00c11d-38f9-430b-92a5-2346fd4007bb None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.002686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214d917-08f0-4391-b957-c4308ab5f9bd None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.037737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85352b79-1b83-414e-ad93-2b21e1406da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:55.038558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352b79-1b83-414e-ad93-2b21e1406da3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.038842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352b79-1b83-414e-ad93-2b21e1406da3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.039160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352b79-1b83-414e-ad93-2b21e1406da3 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.039878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85352b79-1b83-414e-ad93-2b21e1406da3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:55.040182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85352b79-1b83-414e-ad93-2b21e1406da3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:55.042126 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85352b79-1b83-414e-ad93-2b21e1406da3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.054026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 387/748] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:55.054837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85352b79-1b83-414e-ad93-2b21e1406da3 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:55.080872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 362/749] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:55.082065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00c11d-38f9-430b-92a5-2346fd4007bb None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:55.109044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:55.129023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:55.163984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85352b79-1b83-414e-ad93-2b21e1406da3 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.191471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00c11d-38f9-430b-92a5-2346fd4007bb None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.199010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd0790-d17d-48c5-a8e1-01cba7bf31d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:55.199812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd0790-d17d-48c5-a8e1-01cba7bf31d3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.200098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd0790-d17d-48c5-a8e1-01cba7bf31d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.200375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd0790-d17d-48c5-a8e1-01cba7bf31d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.207384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd0790-d17d-48c5-a8e1-01cba7bf31d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:55.208836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd0790-d17d-48c5-a8e1-01cba7bf31d3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-a9cd0790-d17d-48c5-a8e1-01cba7bf31d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:55.223446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:55.224346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:55.225282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:55.225282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:55.225282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:55.225282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:55.225282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:55.225282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:55.225282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:55.225282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:55.225282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 388/750] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:55.225799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd0790-d17d-48c5-a8e1-01cba7bf31d3 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:55.234880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53a5ad-2117-4ae8-a555-42236f04ff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:55.235825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a5ad-2117-4ae8-a555-42236f04ff72 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.236295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a5ad-2117-4ae8-a555-42236f04ff72 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.236769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a5ad-2117-4ae8-a555-42236f04ff72 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.237674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53a5ad-2117-4ae8-a555-42236f04ff72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:55.239272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53a5ad-2117-4ae8-a555-42236f04ff72 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.317129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 363/751] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:40:55 2022] POST /identity/v3/services => generated 240 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:55.318356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53a5ad-2117-4ae8-a555-42236f04ff72 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:55.340809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd0790-d17d-48c5-a8e1-01cba7bf31d3 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.373623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:55.389760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaadb4f-34b8-4b24-9db9-b7738b5f4184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:55.390432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaadb4f-34b8-4b24-9db9-b7738b5f4184 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.390746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaadb4f-34b8-4b24-9db9-b7738b5f4184 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.391015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaadb4f-34b8-4b24-9db9-b7738b5f4184 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.399491 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaadb4f-34b8-4b24-9db9-b7738b5f4184 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:55.401555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaadb4f-34b8-4b24-9db9-b7738b5f4184 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-9eaadb4f-34b8-4b24-9db9-b7738b5f4184 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:40:55.419181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:55.420241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:55.421211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:55.421211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:55.421211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:40:55.421211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:40:55.421211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:40:55.421211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:40:55.421211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:40:55.421211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:40:55.421211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:40:55.421211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:55.425321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 389/752] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:55.426282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaadb4f-34b8-4b24-9db9-b7738b5f4184 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:55.457931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:55.477075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53a5ad-2117-4ae8-a555-42236f04ff72 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.518028 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aa87a-94e0-4377-938e-39960c46b7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:55.518909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aa87a-94e0-4377-938e-39960c46b7c7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.519195 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aa87a-94e0-4377-938e-39960c46b7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.519508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aa87a-94e0-4377-938e-39960c46b7c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.536925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4aa87a-94e0-4377-938e-39960c46b7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:55.538385 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4aa87a-94e0-4377-938e-39960c46b7c7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.562276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaadb4f-34b8-4b24-9db9-b7738b5f4184 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-cf4aa87a-94e0-4377-938e-39960c46b7c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:55.562504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:55.563386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:55.564289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:55.564289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:55.564289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:55.564289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:55.564289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 364/753] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:55.564573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aa87a-94e0-4377-938e-39960c46b7c7 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:55.606035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f1d38-a089-4365-b4a4-46758c5dcb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:55.607248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f1d38-a089-4365-b4a4-46758c5dcb77 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.607531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f1d38-a089-4365-b4a4-46758c5dcb77 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.607809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f1d38-a089-4365-b4a4-46758c5dcb77 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.609055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f1d38-a089-4365-b4a4-46758c5dcb77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:55.609423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f1d38-a089-4365-b4a4-46758c5dcb77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:55.611568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f1d38-a089-4365-b4a4-46758c5dcb77 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.628120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 390/754] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:55.629011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f1d38-a089-4365-b4a4-46758c5dcb77 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:55.677214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aa87a-94e0-4377-938e-39960c46b7c7 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.687053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:55.718462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300b569-0720-446c-984c-882bfd7189ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:55.719419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300b569-0720-446c-984c-882bfd7189ac None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.719911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300b569-0720-446c-984c-882bfd7189ac None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.720382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300b569-0720-446c-984c-882bfd7189ac None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.721411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300b569-0720-446c-984c-882bfd7189ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:55.721945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300b569-0720-446c-984c-882bfd7189ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:55.723761 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300b569-0720-446c-984c-882bfd7189ac None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.728944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:55.751619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 365/755] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 05:40:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:55.752883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300b569-0720-446c-984c-882bfd7189ac None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:55.759341 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f1d38-a089-4365-b4a4-46758c5dcb77 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.809029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6530fa-05f0-4e88-88dc-aeb8ff08b9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:55.809705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6530fa-05f0-4e88-88dc-aeb8ff08b9bc None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.809984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6530fa-05f0-4e88-88dc-aeb8ff08b9bc None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.810262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6530fa-05f0-4e88-88dc-aeb8ff08b9bc None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.811302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6530fa-05f0-4e88-88dc-aeb8ff08b9bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:55.811609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6530fa-05f0-4e88-88dc-aeb8ff08b9bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:55.813394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6530fa-05f0-4e88-88dc-aeb8ff08b9bc None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.830453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 391/756] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:55.831631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6530fa-05f0-4e88-88dc-aeb8ff08b9bc None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:55.870815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300b569-0720-446c-984c-882bfd7189ac None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.908242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a220eaf-7058-43fd-8663-a6ab53f49c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:55.909075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a220eaf-7058-43fd-8663-a6ab53f49c41 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.909513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a220eaf-7058-43fd-8663-a6ab53f49c41 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.909790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a220eaf-7058-43fd-8663-a6ab53f49c41 None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/067b827625e0427294b93b027597a5f3/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.912001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a220eaf-7058-43fd-8663-a6ab53f49c41 None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=067b827625e0427294b93b027597a5f3, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:55.913724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a220eaf-7058-43fd-8663-a6ab53f49c41 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.953152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6530fa-05f0-4e88-88dc-aeb8ff08b9bc None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.956209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:55.998494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 366/757] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:55 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/067b827625e0427294b93b027597a5f3/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:55.998494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a220eaf-7058-43fd-8663-a6ab53f49c41 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.006005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2801cb3-c0f5-4c1e-83ab-bb199792a095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.008224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2801cb3-c0f5-4c1e-83ab-bb199792a095 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.008903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2801cb3-c0f5-4c1e-83ab-bb199792a095 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.009631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2801cb3-c0f5-4c1e-83ab-bb199792a095 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.112930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.155625 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2801cb3-c0f5-4c1e-83ab-bb199792a095 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.157283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2801cb3-c0f5-4c1e-83ab-bb199792a095 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.199696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a220eaf-7058-43fd-8663-a6ab53f49c41 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.243619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43945e40-5b5c-43a5-b0cb-c0b7995f2072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.244344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43945e40-5b5c-43a5-b0cb-c0b7995f2072 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.245208 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43945e40-5b5c-43a5-b0cb-c0b7995f2072 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.245486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43945e40-5b5c-43a5-b0cb-c0b7995f2072 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.253926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43945e40-5b5c-43a5-b0cb-c0b7995f2072 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.255760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43945e40-5b5c-43a5-b0cb-c0b7995f2072 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-43945e40-5b5c-43a5-b0cb-c0b7995f2072 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:56.273704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:56.274751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:56.275700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:56.275700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:56.275700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:56.275700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:56.275700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:56.275700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:56.275700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:56.275700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:56.276175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 367/758] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:56.281644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43945e40-5b5c-43a5-b0cb-c0b7995f2072 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.298437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 392/759] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:40:56.306577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2801cb3-c0f5-4c1e-83ab-bb199792a095 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.310828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.446368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43945e40-5b5c-43a5-b0cb-c0b7995f2072 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.478484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.492648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2801cb3-c0f5-4c1e-83ab-bb199792a095 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.518376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886ef33-8c1e-47f6-8e23-6ddcebf22ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.518989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ef33-8c1e-47f6-8e23-6ddcebf22ae8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.519271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ef33-8c1e-47f6-8e23-6ddcebf22ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.519542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ef33-8c1e-47f6-8e23-6ddcebf22ae8 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.521151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886ef33-8c1e-47f6-8e23-6ddcebf22ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:56.521460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886ef33-8c1e-47f6-8e23-6ddcebf22ae8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.522922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886ef33-8c1e-47f6-8e23-6ddcebf22ae8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.547099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 368/760] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:56.551296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886ef33-8c1e-47f6-8e23-6ddcebf22ae8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.560638 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f8c22-8fba-4664-b050-06131660b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.562694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f8c22-8fba-4664-b050-06131660b6df None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.562975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f8c22-8fba-4664-b050-06131660b6df None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.563245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f8c22-8fba-4664-b050-06131660b6df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.582290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f8c22-8fba-4664-b050-06131660b6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.584087 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f8c22-8fba-4664-b050-06131660b6df None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-6c6f8c22-8fba-4664-b050-06131660b6df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:56.618693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:56.620236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:56.621157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:56.621157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:56.621157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:56.621157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:56.623544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 393/761] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:56.632754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f8c22-8fba-4664-b050-06131660b6df None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.692740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.738115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886ef33-8c1e-47f6-8e23-6ddcebf22ae8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.745320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f8c22-8fba-4664-b050-06131660b6df None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.780861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba5041-1cbc-4969-be22-a9a2527a6593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.781793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba5041-1cbc-4969-be22-a9a2527a6593 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.782426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba5041-1cbc-4969-be22-a9a2527a6593 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.782906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba5041-1cbc-4969-be22-a9a2527a6593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.783631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8c3b4-6f3a-4e4a-864a-bc325bcfcb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.784135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba5041-1cbc-4969-be22-a9a2527a6593 None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=56d11b3ed0474760a10e6f94df838297) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:56.784424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8c3b4-6f3a-4e4a-864a-bc325bcfcb53 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.784779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8c3b4-6f3a-4e4a-864a-bc325bcfcb53 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.785003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba5041-1cbc-4969-be22-a9a2527a6593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.785118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8c3b4-6f3a-4e4a-864a-bc325bcfcb53 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.785917 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8c3b4-6f3a-4e4a-864a-bc325bcfcb53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:56.786290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8c3b4-6f3a-4e4a-864a-bc325bcfcb53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.787115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba5041-1cbc-4969-be22-a9a2527a6593 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.787844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8c3b4-6f3a-4e4a-864a-bc325bcfcb53 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.798503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 369/762] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/role_assignments?user.id=56d11b3ed0474760a10e6f94df838297&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:56.802270 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4a72f-203b-4c67-a332-728156353288 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.802597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4a72f-203b-4c67-a332-728156353288 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.802880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4a72f-203b-4c67-a332-728156353288 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.810763 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 370/763] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:56.812159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb669dd-28f9-4f48-9955-b741a2579101 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.812447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb669dd-28f9-4f48-9955-b741a2579101 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.812725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb669dd-28f9-4f48-9955-b741a2579101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.815522 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 394/764] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:56.821416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8c3b4-6f3a-4e4a-864a-bc325bcfcb53 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.847742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0fb669dd-28f9-4f48-9955-b741a2579101 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:56.848978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0fb669dd-28f9-4f48-9955-b741a2579101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:56.922393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:57.007018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 371/765] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:40:57.013448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8c3b4-6f3a-4e4a-864a-bc325bcfcb53 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.031461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb669dd-28f9-4f48-9955-b741a2579101 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.037043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:57.055991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847a42b-14f3-4a0a-8cb4-ca249107588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.056793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847a42b-14f3-4a0a-8cb4-ca249107588f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.057154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847a42b-14f3-4a0a-8cb4-ca249107588f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.057429 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847a42b-14f3-4a0a-8cb4-ca249107588f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.064993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847a42b-14f3-4a0a-8cb4-ca249107588f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.066697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847a42b-14f3-4a0a-8cb4-ca249107588f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-d847a42b-14f3-4a0a-8cb4-ca249107588f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:40:57.097859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:57.099131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:57.100015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:40:57.100015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:40:57.100015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:40:57.100015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:40:57.100015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:40:57.100015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:40:57.100015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:40:57.100015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:57.107479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 395/766] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:57.114901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847a42b-14f3-4a0a-8cb4-ca249107588f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.170257 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb669dd-28f9-4f48-9955-b741a2579101 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.239774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff21d8-06ae-4412-a47a-007cbb653df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.240619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff21d8-06ae-4412-a47a-007cbb653df0 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.240917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff21d8-06ae-4412-a47a-007cbb653df0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.241152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff21d8-06ae-4412-a47a-007cbb653df0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.241811 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff21d8-06ae-4412-a47a-007cbb653df0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.243333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff21d8-06ae-4412-a47a-007cbb653df0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-2aff21d8-06ae-4412-a47a-007cbb653df0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:57.271127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:57.272137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:57.272995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 372/767] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:57.278983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff21d8-06ae-4412-a47a-007cbb653df0 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.299228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847a42b-14f3-4a0a-8cb4-ca249107588f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.342848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:57.343353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830bad7-089e-46ab-aace-3423b1e78e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.344131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830bad7-089e-46ab-aace-3423b1e78e96 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.344273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830bad7-089e-46ab-aace-3423b1e78e96 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.344537 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830bad7-089e-46ab-aace-3423b1e78e96 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.345339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830bad7-089e-46ab-aace-3423b1e78e96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:57.345649 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830bad7-089e-46ab-aace-3423b1e78e96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.347153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830bad7-089e-46ab-aace-3423b1e78e96 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:57.381065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 396/768] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:57.389473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830bad7-089e-46ab-aace-3423b1e78e96 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.439516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff21d8-06ae-4412-a47a-007cbb653df0 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.476965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582c2d0-4362-4dbc-8cf6-f809b7796625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.477654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582c2d0-4362-4dbc-8cf6-f809b7796625 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.477944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582c2d0-4362-4dbc-8cf6-f809b7796625 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.478227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582c2d0-4362-4dbc-8cf6-f809b7796625 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.479519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582c2d0-4362-4dbc-8cf6-f809b7796625 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:57.480060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582c2d0-4362-4dbc-8cf6-f809b7796625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.481352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582c2d0-4362-4dbc-8cf6-f809b7796625 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:57.486862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 373/769] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:57.493836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582c2d0-4362-4dbc-8cf6-f809b7796625 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.495612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:57.586146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830bad7-089e-46ab-aace-3423b1e78e96 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.625823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c502ed0-d110-4661-8b38-9cc95f61fd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.626866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c502ed0-d110-4661-8b38-9cc95f61fd3d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.627152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c502ed0-d110-4661-8b38-9cc95f61fd3d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.627706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c502ed0-d110-4661-8b38-9cc95f61fd3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.628562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c502ed0-d110-4661-8b38-9cc95f61fd3d None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=5c359fc0168141fb8da3f953884fd2e5) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:57.628879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c502ed0-d110-4661-8b38-9cc95f61fd3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.631123 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c502ed0-d110-4661-8b38-9cc95f61fd3d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:57.639285 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 397/770] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/role_assignments?user.id=5c359fc0168141fb8da3f953884fd2e5&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=7a833c45603a4830b888e58e1232712a => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:57.640749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fd75b-531c-44c8-a18b-a27e159d6bd3 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.641034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fd75b-531c-44c8-a18b-a27e159d6bd3 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.641306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fd75b-531c-44c8-a18b-a27e159d6bd3 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.647794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 398/771] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:57.649673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bb937-3418-46f1-b90d-9727c56168a3 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.650073 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bb937-3418-46f1-b90d-9727c56168a3 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.650354 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bb937-3418-46f1-b90d-9727c56168a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.669937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a38bb937-3418-46f1-b90d-9727c56168a3 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:57.671234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a38bb937-3418-46f1-b90d-9727c56168a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:57.685039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:57.691622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582c2d0-4362-4dbc-8cf6-f809b7796625 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.733513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a66ea-6296-4616-aa64-ccc32b7e65d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.734465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a66ea-6296-4616-aa64-ccc32b7e65d3 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.734843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a66ea-6296-4616-aa64-ccc32b7e65d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.735507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a66ea-6296-4616-aa64-ccc32b7e65d3 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.736311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a66ea-6296-4616-aa64-ccc32b7e65d3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:57.736700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a66ea-6296-4616-aa64-ccc32b7e65d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.738496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a66ea-6296-4616-aa64-ccc32b7e65d3 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:57.746847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 374/772] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:57.753186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a66ea-6296-4616-aa64-ccc32b7e65d3 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.831482 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 399/773] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:40:57.855744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bb937-3418-46f1-b90d-9727c56168a3 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.894856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:57.901562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a66ea-6296-4616-aa64-ccc32b7e65d3 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.943875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56724ce8-40ad-4e09-89db-e423f21ba3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.945008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56724ce8-40ad-4e09-89db-e423f21ba3ec None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.945442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56724ce8-40ad-4e09-89db-e423f21ba3ec None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.945731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56724ce8-40ad-4e09-89db-e423f21ba3ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.946558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56724ce8-40ad-4e09-89db-e423f21ba3ec None admin] RBAC: Adding query filter params (interface=public, service_id=78d03815b4334fad9002dec4450ef9e9, region_id=RegionOne) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:57.947028 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56724ce8-40ad-4e09-89db-e423f21ba3ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.948434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56724ce8-40ad-4e09-89db-e423f21ba3ec None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:57.957403 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 375/774] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/endpoints?service_id=78d03815b4334fad9002dec4450ef9e9&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:57.962906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56724ce8-40ad-4e09-89db-e423f21ba3ec None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.992910 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:58.076858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56724ce8-40ad-4e09-89db-e423f21ba3ec None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:58.092428 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bb937-3418-46f1-b90d-9727c56168a3 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:58.115750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee18dee-f60c-411c-9131-c67124c589f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:58.116880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18dee-f60c-411c-9131-c67124c589f1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.117240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18dee-f60c-411c-9131-c67124c589f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.117591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee18dee-f60c-411c-9131-c67124c589f1 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.118353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee18dee-f60c-411c-9131-c67124c589f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:58.119996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee18dee-f60c-411c-9131-c67124c589f1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:58.125921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 376/775] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:58.153855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd70e0f-18e4-4d3d-8c77-3dbdc2fb6fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:58.154723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd70e0f-18e4-4d3d-8c77-3dbdc2fb6fb8 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.155003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd70e0f-18e4-4d3d-8c77-3dbdc2fb6fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.155276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd70e0f-18e4-4d3d-8c77-3dbdc2fb6fb8 None admin] PATH_INFO: `/v3/services/network` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.155948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd70e0f-18e4-4d3d-8c77-3dbdc2fb6fb8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:58.157370 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd70e0f-18e4-4d3d-8c77-3dbdc2fb6fb8 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-3dd70e0f-18e4-4d3d-8c77-3dbdc2fb6fb8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:58.170912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:58.171997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:40:58.172874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:58.177049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 400/776] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/services/network => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:58.178389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee18dee-f60c-411c-9131-c67124c589f1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:58.310707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:58.327403 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee18dee-f60c-411c-9131-c67124c589f1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:58.366372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8423f-b65f-4b44-981a-0044ab172c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:58.366998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8423f-b65f-4b44-981a-0044ab172c88 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.367274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8423f-b65f-4b44-981a-0044ab172c88 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.367547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8423f-b65f-4b44-981a-0044ab172c88 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.368245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8423f-b65f-4b44-981a-0044ab172c88 None admin] RBAC: Adding query filter params (name=network) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:58.368546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8423f-b65f-4b44-981a-0044ab172c88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:58.370088 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8423f-b65f-4b44-981a-0044ab172c88 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:58.378608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 377/777] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:40:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:58.380607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd70e0f-18e4-4d3d-8c77-3dbdc2fb6fb8 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:58.479903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:58.486766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd70e0f-18e4-4d3d-8c77-3dbdc2fb6fb8 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:58.511200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327756-91f4-4b42-9b6e-d025fa10502f None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.511936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327756-91f4-4b42-9b6e-d025fa10502f None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.512926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327756-91f4-4b42-9b6e-d025fa10502f None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.516608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 378/778] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:58.522613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5e857-dbb0-4494-8d11-4a84df22187d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.522894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5e857-dbb0-4494-8d11-4a84df22187d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.523202 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5e857-dbb0-4494-8d11-4a84df22187d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.537807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc380ee-de76-43d2-870b-0c97f13be138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:58.538502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc380ee-de76-43d2-870b-0c97f13be138 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.538812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc380ee-de76-43d2-870b-0c97f13be138 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.539090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc380ee-de76-43d2-870b-0c97f13be138 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.539807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc380ee-de76-43d2-870b-0c97f13be138 None admin] RBAC: Adding query filter params (type=network) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:58.540114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc380ee-de76-43d2-870b-0c97f13be138 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:58.541525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc380ee-de76-43d2-870b-0c97f13be138 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:58.543021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-6ad5e857-dbb0-4494-8d11-4a84df22187d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:58.544223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6ad5e857-dbb0-4494-8d11-4a84df22187d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:58.555208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 401/779] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:40:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:58.561115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc380ee-de76-43d2-870b-0c97f13be138 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:58.629525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 379/780] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:40:58.652142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5e857-dbb0-4494-8d11-4a84df22187d None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:58.740453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc380ee-de76-43d2-870b-0c97f13be138 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:58.787751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:58.808812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604e17c-0c9f-4e69-8385-00788f7a5a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:58.809482 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604e17c-0c9f-4e69-8385-00788f7a5a2e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.809762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604e17c-0c9f-4e69-8385-00788f7a5a2e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.810035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604e17c-0c9f-4e69-8385-00788f7a5a2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.810864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604e17c-0c9f-4e69-8385-00788f7a5a2e None admin] RBAC: Adding query filter params (interface=public, service_id=3f6ca764585a4eaf93c91db4905e7b74, region_id=RegionOne) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:58.811172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604e17c-0c9f-4e69-8385-00788f7a5a2e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:58.812548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604e17c-0c9f-4e69-8385-00788f7a5a2e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:58.824593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5e857-dbb0-4494-8d11-4a84df22187d None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:58.835667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 402/781] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 05:40:58 2022] GET /identity/v3/endpoints?service_id=3f6ca764585a4eaf93c91db4905e7b74&interface=public®ion_id=RegionOne => generated 194 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:58.836538 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604e17c-0c9f-4e69-8385-00788f7a5a2e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:58.883027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:58.888600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13841d55-e9f0-486f-8190-6a96263d7383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:58.889660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13841d55-e9f0-486f-8190-6a96263d7383 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.889939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13841d55-e9f0-486f-8190-6a96263d7383 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.896416 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13841d55-e9f0-486f-8190-6a96263d7383 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.915885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13841d55-e9f0-486f-8190-6a96263d7383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-13841d55-e9f0-486f-8190-6a96263d7383 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:58.925266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:58.927299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:58.929157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 380/782] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:58.934942 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13841d55-e9f0-486f-8190-6a96263d7383 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.001705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604e17c-0c9f-4e69-8385-00788f7a5a2e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.064057 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303c4ed-6438-4194-9621-6c7a07ccfa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.064724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303c4ed-6438-4194-9621-6c7a07ccfa7b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.065000 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303c4ed-6438-4194-9621-6c7a07ccfa7b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.065274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303c4ed-6438-4194-9621-6c7a07ccfa7b None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.066879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303c4ed-6438-4194-9621-6c7a07ccfa7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:59.068256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303c4ed-6438-4194-9621-6c7a07ccfa7b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:59.079692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 403/783] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 05:40:58 2022] GET /identity/v3/services => generated 772 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:59.120734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13841d55-e9f0-486f-8190-6a96263d7383 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.179767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9bba7-388d-42f3-b261-107e6f71f0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.180377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9bba7-388d-42f3-b261-107e6f71f0a4 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.180663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9bba7-388d-42f3-b261-107e6f71f0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.180946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9bba7-388d-42f3-b261-107e6f71f0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.181715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9bba7-388d-42f3-b261-107e6f71f0a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:59.182069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9bba7-388d-42f3-b261-107e6f71f0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:59.183510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9bba7-388d-42f3-b261-107e6f71f0a4 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:59.193073 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 381/784] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:59.199464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303c4ed-6438-4194-9621-6c7a07ccfa7b None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.310847 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.323743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303c4ed-6438-4194-9621-6c7a07ccfa7b None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.323904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55002355-7844-476c-a431-23eddafa07c5 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.324019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55002355-7844-476c-a431-23eddafa07c5 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.324292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55002355-7844-476c-a431-23eddafa07c5 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.325815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 382/785] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:40:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:59.330673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be98738-822f-4ea9-a1a0-fc7eb8ce68f9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.330966 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be98738-822f-4ea9-a1a0-fc7eb8ce68f9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.343874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be98738-822f-4ea9-a1a0-fc7eb8ce68f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.343874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.351975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-7be98738-822f-4ea9-a1a0-fc7eb8ce68f9 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:59.352723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7be98738-822f-4ea9-a1a0-fc7eb8ce68f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:59.364949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cb431-7686-41e7-be19-ca29aaacb9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.366001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cb431-7686-41e7-be19-ca29aaacb9e1 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.366492 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cb431-7686-41e7-be19-ca29aaacb9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.366908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cb431-7686-41e7-be19-ca29aaacb9e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.377290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cb431-7686-41e7-be19-ca29aaacb9e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:59.379031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cb431-7686-41e7-be19-ca29aaacb9e1 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-fe3cb431-7686-41e7-be19-ca29aaacb9e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:59.401949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:59.403084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:59.404207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:59.404207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:59.404207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:59.404207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:40:59.410637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 404/786] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:59.411958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cb431-7686-41e7-be19-ca29aaacb9e1 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.425729 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 383/787] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:40:59.451842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be98738-822f-4ea9-a1a0-fc7eb8ce68f9 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.562045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be98738-822f-4ea9-a1a0-fc7eb8ce68f9 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.599458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398b7d0-eb29-43ec-baf2-198560761409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.600617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398b7d0-eb29-43ec-baf2-198560761409 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.600895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398b7d0-eb29-43ec-baf2-198560761409 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.601168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398b7d0-eb29-43ec-baf2-198560761409 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.606770 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.615187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398b7d0-eb29-43ec-baf2-198560761409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-f398b7d0-eb29-43ec-baf2-198560761409 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:59.630085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:59.631326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:40:59.632389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:40:59.634636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 384/788] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:59.641079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398b7d0-eb29-43ec-baf2-198560761409 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.666182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cb431-7686-41e7-be19-ca29aaacb9e1 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.740385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458d56e-7665-4159-bc6f-6b841bfda34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.740991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458d56e-7665-4159-bc6f-6b841bfda34c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.741267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458d56e-7665-4159-bc6f-6b841bfda34c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.741536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458d56e-7665-4159-bc6f-6b841bfda34c None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.742279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458d56e-7665-4159-bc6f-6b841bfda34c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:59.742634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458d56e-7665-4159-bc6f-6b841bfda34c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:59.744928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458d56e-7665-4159-bc6f-6b841bfda34c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:59.762879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.768180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 405/789] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:59.772469 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458d56e-7665-4159-bc6f-6b841bfda34c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.855266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398b7d0-eb29-43ec-baf2-198560761409 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.938160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458d56e-7665-4159-bc6f-6b841bfda34c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.972799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56d07-aa93-45c2-bbbf-2c62d3f17e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.973650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56d07-aa93-45c2-bbbf-2c62d3f17e94 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.973931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56d07-aa93-45c2-bbbf-2c62d3f17e94 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.974232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353b31a-27ea-4dc1-bc5d-a3eabfba8210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.974366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56d07-aa93-45c2-bbbf-2c62d3f17e94 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.975445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353b31a-27ea-4dc1-bc5d-a3eabfba8210 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.975924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353b31a-27ea-4dc1-bc5d-a3eabfba8210 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.976390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353b31a-27ea-4dc1-bc5d-a3eabfba8210 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.977328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353b31a-27ea-4dc1-bc5d-a3eabfba8210 None admin] RBAC: Adding query filter params (name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:59.977928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353b31a-27ea-4dc1-bc5d-a3eabfba8210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:59.984114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a56d07-aa93-45c2-bbbf-2c62d3f17e94 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:59.984114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a56d07-aa93-45c2-bbbf-2c62d3f17e94 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:59.985615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353b31a-27ea-4dc1-bc5d-a3eabfba8210 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-30a56d07-aa93-45c2-bbbf-2c62d3f17e94 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:00.000515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:00.001655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:00.002877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:00.002877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:00.002877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:00.002877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:00.002877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:00.002877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:00.002877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:00.002877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:00.002877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:00.002877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:00.004268 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 406/790] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:00.005681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 385/791] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:00.011887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353b31a-27ea-4dc1-bc5d-a3eabfba8210 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:00.012746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56d07-aa93-45c2-bbbf-2c62d3f17e94 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:00.078935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:00.089179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:00.120684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56d07-aa93-45c2-bbbf-2c62d3f17e94 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:00.143688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353b31a-27ea-4dc1-bc5d-a3eabfba8210 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:00.159309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e30fd-7c03-4e47-bb1a-c31a3d0b9855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:00.160233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e30fd-7c03-4e47-bb1a-c31a3d0b9855 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.160615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e30fd-7c03-4e47-bb1a-c31a3d0b9855 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.160984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e30fd-7c03-4e47-bb1a-c31a3d0b9855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.173019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e30fd-7c03-4e47-bb1a-c31a3d0b9855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:00.174838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e30fd-7c03-4e47-bb1a-c31a3d0b9855 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:00.187344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7703e-e3ae-4aee-9788-49e7b99ce8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:00.188219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7703e-e3ae-4aee-9788-49e7b99ce8dc None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.188644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7703e-e3ae-4aee-9788-49e7b99ce8dc None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.189063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7703e-e3ae-4aee-9788-49e7b99ce8dc None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.190032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7703e-e3ae-4aee-9788-49e7b99ce8dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:00.190521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7703e-e3ae-4aee-9788-49e7b99ce8dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:00.192089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7703e-e3ae-4aee-9788-49e7b99ce8dc None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-ee1e30fd-7c03-4e47-bb1a-c31a3d0b9855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:00.194158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:00.195152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:00.196054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:00.196054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:00.196054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:00.196054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:00.196286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 407/792] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:00.198159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac464d5c-a74f-4061-a420-2933b987cfda None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.198402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac464d5c-a74f-4061-a420-2933b987cfda None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.198779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac464d5c-a74f-4061-a420-2933b987cfda None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.200686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 408/793] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:00.202084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac464d5c-a74f-4061-a420-2933b987cfda None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:00.216142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 386/794] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 05:41:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:00.220753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9121c2-1e9f-446f-8c2d-3a2e2bc1d829 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.220753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9121c2-1e9f-446f-8c2d-3a2e2bc1d829 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.220753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9121c2-1e9f-446f-8c2d-3a2e2bc1d829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.241987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-fa9121c2-1e9f-446f-8c2d-3a2e2bc1d829 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:00.243075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-fa9121c2-1e9f-446f-8c2d-3a2e2bc1d829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:00.343660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac464d5c-a74f-4061-a420-2933b987cfda None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:00.357063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:00.383258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68950a89-8dff-417c-955e-668762df9e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:00.384292 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68950a89-8dff-417c-955e-668762df9e78 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.386128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68950a89-8dff-417c-955e-668762df9e78 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.386128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68950a89-8dff-417c-955e-668762df9e78 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.386128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68950a89-8dff-417c-955e-668762df9e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:00.386663 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68950a89-8dff-417c-955e-668762df9e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:00.388285 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68950a89-8dff-417c-955e-668762df9e78 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:00.402220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 409/795] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:00.403514 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68950a89-8dff-417c-955e-668762df9e78 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:00.443871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 387/796] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:00.445635 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9121c2-1e9f-446f-8c2d-3a2e2bc1d829 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:00.530525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:00.537301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68950a89-8dff-417c-955e-668762df9e78 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:00.575911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6cd50-603a-44b8-94d8-6f85c7082fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:00.576672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6cd50-603a-44b8-94d8-6f85c7082fba None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.576956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6cd50-603a-44b8-94d8-6f85c7082fba None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.577338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6cd50-603a-44b8-94d8-6f85c7082fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.597494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9121c2-1e9f-446f-8c2d-3a2e2bc1d829 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:00.620119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:00.647347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3bf41-a480-407a-9905-2ce67c179d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:00.648170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3bf41-a480-407a-9905-2ce67c179d30 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.648454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3bf41-a480-407a-9905-2ce67c179d30 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.648892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3bf41-a480-407a-9905-2ce67c179d30 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.651942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6cd50-603a-44b8-94d8-6f85c7082fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:00.653734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6cd50-603a-44b8-94d8-6f85c7082fba None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:00.659524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3bf41-a480-407a-9905-2ce67c179d30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:00.661082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3bf41-a480-407a-9905-2ce67c179d30 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-eca3bf41-a480-407a-9905-2ce67c179d30 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:00.681470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:00.683041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:00.683996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:00.683996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:00.683996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:00.683996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:00.683996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:00.683996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:00.683996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:00.683996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:00.683996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:00.683996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:00.684539 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 388/797] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 05:41:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:00.690564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3bf41-a480-407a-9905-2ce67c179d30 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:00.762764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 410/798] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:00.763615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6cd50-603a-44b8-94d8-6f85c7082fba None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:00.803262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3bf41-a480-407a-9905-2ce67c179d30 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:00.806274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:00.839831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd64ea2-4e96-4e88-a1c5-93d2fcef534d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:00.840625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd64ea2-4e96-4e88-a1c5-93d2fcef534d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.840902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd64ea2-4e96-4e88-a1c5-93d2fcef534d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.841170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd64ea2-4e96-4e88-a1c5-93d2fcef534d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.844612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:00.852717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd64ea2-4e96-4e88-a1c5-93d2fcef534d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-efd64ea2-4e96-4e88-a1c5-93d2fcef534d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:00.860955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:00.861913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:00.862824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 389/799] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:00.863616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53110a2-611c-4121-8562-986bd6fdb9dd None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.863703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53110a2-611c-4121-8562-986bd6fdb9dd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.864107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53110a2-611c-4121-8562-986bd6fdb9dd None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.869143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 390/800] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:00.870552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53110a2-611c-4121-8562-986bd6fdb9dd None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:00.881243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6cd50-603a-44b8-94d8-6f85c7082fba None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:00.927675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71afa0d-d8e9-473b-ba4e-43e415e15341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:00.928468 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71afa0d-d8e9-473b-ba4e-43e415e15341 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.928757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71afa0d-d8e9-473b-ba4e-43e415e15341 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.929034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71afa0d-d8e9-473b-ba4e-43e415e15341 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.929870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71afa0d-d8e9-473b-ba4e-43e415e15341 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:00.930186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71afa0d-d8e9-473b-ba4e-43e415e15341 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:00.931645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71afa0d-d8e9-473b-ba4e-43e415e15341 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:00.954802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 411/801] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 05:41:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:00.954802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71afa0d-d8e9-473b-ba4e-43e415e15341 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:00.985152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53110a2-611c-4121-8562-986bd6fdb9dd None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.027005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7f3e1-06b2-4d00-a776-d4bbec94d7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.028365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7f3e1-06b2-4d00-a776-d4bbec94d7b5 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.028917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7f3e1-06b2-4d00-a776-d4bbec94d7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.029205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7f3e1-06b2-4d00-a776-d4bbec94d7b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.042651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7f3e1-06b2-4d00-a776-d4bbec94d7b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.044806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7f3e1-06b2-4d00-a776-d4bbec94d7b5 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.060953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71afa0d-d8e9-473b-ba4e-43e415e15341 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-92b7f3e1-06b2-4d00-a776-d4bbec94d7b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:01.067364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:01.068266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:01.074191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:01.074191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:01.074191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:01.074191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:01.074191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 391/802] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:01.074191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed784f-05fd-400b-ab93-b2044c3edfe3 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.074191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed784f-05fd-400b-ab93-b2044c3edfe3 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.074191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed784f-05fd-400b-ab93-b2044c3edfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.087098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:01.094150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-77ed784f-05fd-400b-ab93-b2044c3edfe3 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:01.095128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bebe2d-a324-4338-8a07-1873ee780bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.095495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-77ed784f-05fd-400b-ab93-b2044c3edfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:01.095808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bebe2d-a324-4338-8a07-1873ee780bc7 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.102596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bebe2d-a324-4338-8a07-1873ee780bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.102596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bebe2d-a324-4338-8a07-1873ee780bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.102596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bebe2d-a324-4338-8a07-1873ee780bc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:01.102596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bebe2d-a324-4338-8a07-1873ee780bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.102596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bebe2d-a324-4338-8a07-1873ee780bc7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.108462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 412/803] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:01.109253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bebe2d-a324-4338-8a07-1873ee780bc7 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:01.110051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:01.186505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 392/804] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:01.186505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed784f-05fd-400b-ab93-b2044c3edfe3 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:01.229872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bebe2d-a324-4338-8a07-1873ee780bc7 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.262410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd9432-8fa4-4d2b-8d03-d539112d6481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.263251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9432-8fa4-4d2b-8d03-d539112d6481 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.263534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9432-8fa4-4d2b-8d03-d539112d6481 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.264078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9432-8fa4-4d2b-8d03-d539112d6481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.274713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd9432-8fa4-4d2b-8d03-d539112d6481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.276718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd9432-8fa4-4d2b-8d03-d539112d6481 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.288865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed784f-05fd-400b-ab93-b2044c3edfe3 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-a4dd9432-8fa4-4d2b-8d03-d539112d6481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:01.294152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:01.295085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:01.301171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:01.301171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:01.301171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:01.301171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:01.301171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 413/805] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:01.301171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd9432-8fa4-4d2b-8d03-d539112d6481 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:01.329866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b5e3-6d5a-4424-89c8-891162dad0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.330870 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b5e3-6d5a-4424-89c8-891162dad0b1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.331327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b5e3-6d5a-4424-89c8-891162dad0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.331600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b5e3-6d5a-4424-89c8-891162dad0b1 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.332320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b5e3-6d5a-4424-89c8-891162dad0b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:01.332623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b5e3-6d5a-4424-89c8-891162dad0b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.333981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b5e3-6d5a-4424-89c8-891162dad0b1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.345957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:01.346508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 393/806] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:01.348032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b5e3-6d5a-4424-89c8-891162dad0b1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:01.364664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:01.418772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd9432-8fa4-4d2b-8d03-d539112d6481 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.451937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c1f3c-ccdb-4ffb-ada9-42898bc47d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.452726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c1f3c-ccdb-4ffb-ada9-42898bc47d6f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.452874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b5e3-6d5a-4424-89c8-891162dad0b1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.453160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c1f3c-ccdb-4ffb-ada9-42898bc47d6f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.453317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c1f3c-ccdb-4ffb-ada9-42898bc47d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.465238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c1f3c-ccdb-4ffb-ada9-42898bc47d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.466715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c1f3c-ccdb-4ffb-ada9-42898bc47d6f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-087c1f3c-ccdb-4ffb-ada9-42898bc47d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:01.483161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:01.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:01.485305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:01.485305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:01.485305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:01.485305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:01.485305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 414/807] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:01.485655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c1f3c-ccdb-4ffb-ada9-42898bc47d6f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:01.489175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20740ed8-8ac8-48ec-a52b-850e8975deba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.490126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20740ed8-8ac8-48ec-a52b-850e8975deba None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.490450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20740ed8-8ac8-48ec-a52b-850e8975deba None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.490751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20740ed8-8ac8-48ec-a52b-850e8975deba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.496471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20740ed8-8ac8-48ec-a52b-850e8975deba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.496471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20740ed8-8ac8-48ec-a52b-850e8975deba None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-20740ed8-8ac8-48ec-a52b-850e8975deba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:01.505160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:01.506352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:01.507483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 394/808] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:01.508797 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20740ed8-8ac8-48ec-a52b-850e8975deba None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:01.573837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:01.580854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:01.586984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c1f3c-ccdb-4ffb-ada9-42898bc47d6f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.604159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20740ed8-8ac8-48ec-a52b-850e8975deba None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.619700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89301084-5d6e-4fcb-887c-1c1bd4ff81b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.620298 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89301084-5d6e-4fcb-887c-1c1bd4ff81b2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.620577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89301084-5d6e-4fcb-887c-1c1bd4ff81b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.620852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89301084-5d6e-4fcb-887c-1c1bd4ff81b2 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.621562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89301084-5d6e-4fcb-887c-1c1bd4ff81b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:01.621862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89301084-5d6e-4fcb-887c-1c1bd4ff81b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.623310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89301084-5d6e-4fcb-887c-1c1bd4ff81b2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.640232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 415/809] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:01.640232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b535b5-e83f-4171-b14f-7ad09779bac9 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.640232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b535b5-e83f-4171-b14f-7ad09779bac9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.640232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b535b5-e83f-4171-b14f-7ad09779bac9 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.640659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e290a1-713a-4d48-b0cc-8eb64d576f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.640764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 416/810] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:01.640849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e290a1-713a-4d48-b0cc-8eb64d576f55 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.640949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b535b5-e83f-4171-b14f-7ad09779bac9 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:01.641035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e290a1-713a-4d48-b0cc-8eb64d576f55 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.641035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e290a1-713a-4d48-b0cc-8eb64d576f55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.649673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e290a1-713a-4d48-b0cc-8eb64d576f55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.651432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e290a1-713a-4d48-b0cc-8eb64d576f55 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-a8e290a1-713a-4d48-b0cc-8eb64d576f55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:01.665652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:01.666724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:01.667685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:01.667685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:01.667685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:01.667685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:01.667685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:01.667685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:01.667685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:01.667685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:01.667685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 395/811] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:01.667685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e290a1-713a-4d48-b0cc-8eb64d576f55 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:01.750372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b535b5-e83f-4171-b14f-7ad09779bac9 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.768127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e290a1-713a-4d48-b0cc-8eb64d576f55 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.784318 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7230a0-26f3-46b1-801b-de4717caf746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.784976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7230a0-26f3-46b1-801b-de4717caf746 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.785254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7230a0-26f3-46b1-801b-de4717caf746 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.785545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7230a0-26f3-46b1-801b-de4717caf746 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.786238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7230a0-26f3-46b1-801b-de4717caf746 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:01.786565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7230a0-26f3-46b1-801b-de4717caf746 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.787933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7230a0-26f3-46b1-801b-de4717caf746 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.796620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:01.798951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 417/812] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:01.799827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7230a0-26f3-46b1-801b-de4717caf746 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:01.802452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1a17e-0b5a-4c93-b930-79d4c2e88595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.803267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1a17e-0b5a-4c93-b930-79d4c2e88595 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.803427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1a17e-0b5a-4c93-b930-79d4c2e88595 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.804193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1a17e-0b5a-4c93-b930-79d4c2e88595 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.804377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1a17e-0b5a-4c93-b930-79d4c2e88595 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:01.804682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1a17e-0b5a-4c93-b930-79d4c2e88595 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.806032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1a17e-0b5a-4c93-b930-79d4c2e88595 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.811118 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 396/813] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:01.812293 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef26e-bd0a-4cd1-af0c-cdbb9713deec None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.812608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef26e-bd0a-4cd1-af0c-cdbb9713deec None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.812880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef26e-bd0a-4cd1-af0c-cdbb9713deec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.819565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:01.829962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-64eef26e-bd0a-4cd1-af0c-cdbb9713deec None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:01.831197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-64eef26e-bd0a-4cd1-af0c-cdbb9713deec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:01.897584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 397/814] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:01.898341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eef26e-bd0a-4cd1-af0c-cdbb9713deec None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:01.905823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7230a0-26f3-46b1-801b-de4717caf746 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:01.941453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de03a49-21de-424a-83bc-856bc3610293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:01.942249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de03a49-21de-424a-83bc-856bc3610293 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.942561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de03a49-21de-424a-83bc-856bc3610293 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.942837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de03a49-21de-424a-83bc-856bc3610293 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.950220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de03a49-21de-424a-83bc-856bc3610293 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:01.951713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de03a49-21de-424a-83bc-856bc3610293 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-0de03a49-21de-424a-83bc-856bc3610293 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:01.965074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:01.965948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:01.966912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:01.966912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:01.966912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:01.966912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:01.966912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:01.966912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:01.966912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:01.966912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:01.966912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 418/815] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:01.967410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de03a49-21de-424a-83bc-856bc3610293 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.009337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eef26e-bd0a-4cd1-af0c-cdbb9713deec None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.033853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.043112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.044677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c5255-86b4-43a9-9b08-c3d2b4540fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.051460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c5255-86b4-43a9-9b08-c3d2b4540fad None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.051460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c5255-86b4-43a9-9b08-c3d2b4540fad None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.051460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c5255-86b4-43a9-9b08-c3d2b4540fad None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.051460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c5255-86b4-43a9-9b08-c3d2b4540fad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:02.051460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c5255-86b4-43a9-9b08-c3d2b4540fad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.051460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c5255-86b4-43a9-9b08-c3d2b4540fad None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.064004 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 398/816] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:02.064804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c5255-86b4-43a9-9b08-c3d2b4540fad None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.072070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de03a49-21de-424a-83bc-856bc3610293 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.105209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4be4b-b9e2-4b77-934b-0fd41dba5ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.106773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4be4b-b9e2-4b77-934b-0fd41dba5ab6 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.107057 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4be4b-b9e2-4b77-934b-0fd41dba5ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.107332 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4be4b-b9e2-4b77-934b-0fd41dba5ab6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.115478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4be4b-b9e2-4b77-934b-0fd41dba5ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.116930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4be4b-b9e2-4b77-934b-0fd41dba5ab6 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-22d4be4b-b9e2-4b77-934b-0fd41dba5ab6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:02.131435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:02.132550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 419/817] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:02.133780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4be4b-b9e2-4b77-934b-0fd41dba5ab6 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.167891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c5255-86b4-43a9-9b08-c3d2b4540fad None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.202093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf442322-f7ce-477f-92e6-1ee48e6f1a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.202786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf442322-f7ce-477f-92e6-1ee48e6f1a10 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.203068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf442322-f7ce-477f-92e6-1ee48e6f1a10 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.203343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf442322-f7ce-477f-92e6-1ee48e6f1a10 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.204039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf442322-f7ce-477f-92e6-1ee48e6f1a10 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:02.204343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf442322-f7ce-477f-92e6-1ee48e6f1a10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.205691 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf442322-f7ce-477f-92e6-1ee48e6f1a10 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.210373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 399/818] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:02.211168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf442322-f7ce-477f-92e6-1ee48e6f1a10 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.234277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4be4b-b9e2-4b77-934b-0fd41dba5ab6 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.261406 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.267464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c1f5a-19a8-4711-9adc-36e05e90602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.268256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c1f5a-19a8-4711-9adc-36e05e90602e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.268536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c1f5a-19a8-4711-9adc-36e05e90602e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.279338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c1f5a-19a8-4711-9adc-36e05e90602e None admin] PATH_INFO: `/v3/services/network` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.279338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c1f5a-19a8-4711-9adc-36e05e90602e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.279338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c1f5a-19a8-4711-9adc-36e05e90602e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.279338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-b66c1f5a-19a8-4711-9adc-36e05e90602e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:02.283024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:02.284250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:02.285122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 420/819] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/services/network => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:02.285999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c1f5a-19a8-4711-9adc-36e05e90602e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.312002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf442322-f7ce-477f-92e6-1ee48e6f1a10 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.346876 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daabc43-8a30-42c7-b85e-bfb847a16d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.347549 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabc43-8a30-42c7-b85e-bfb847a16d8f None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.347829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabc43-8a30-42c7-b85e-bfb847a16d8f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.348103 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabc43-8a30-42c7-b85e-bfb847a16d8f None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.350540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daabc43-8a30-42c7-b85e-bfb847a16d8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:02.350869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daabc43-8a30-42c7-b85e-bfb847a16d8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.352312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daabc43-8a30-42c7-b85e-bfb847a16d8f None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.368210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 400/820] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:02.368801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daabc43-8a30-42c7-b85e-bfb847a16d8f None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.395790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c1f5a-19a8-4711-9adc-36e05e90602e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.429336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f295ca-d519-4a30-ad48-66a9079360c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.429913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f295ca-d519-4a30-ad48-66a9079360c7 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.430192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f295ca-d519-4a30-ad48-66a9079360c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.430487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f295ca-d519-4a30-ad48-66a9079360c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.431320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f295ca-d519-4a30-ad48-66a9079360c7 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=067b827625e0427294b93b027597a5f3) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:02.431627 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f295ca-d519-4a30-ad48-66a9079360c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.433082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f295ca-d519-4a30-ad48-66a9079360c7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.438840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 421/821] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/role_assignments?user.id=067b827625e0427294b93b027597a5f3&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=7a833c45603a4830b888e58e1232712a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:02.439598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f295ca-d519-4a30-ad48-66a9079360c7 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.482238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daabc43-8a30-42c7-b85e-bfb847a16d8f None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.492747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.512337 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.520487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f79f1a-f1a0-46f7-8630-2de5a0b7f39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.521187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79f1a-f1a0-46f7-8630-2de5a0b7f39b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.521482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79f1a-f1a0-46f7-8630-2de5a0b7f39b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.521799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79f1a-f1a0-46f7-8630-2de5a0b7f39b None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.522678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f79f1a-f1a0-46f7-8630-2de5a0b7f39b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:02.522995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f79f1a-f1a0-46f7-8630-2de5a0b7f39b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.524476 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f79f1a-f1a0-46f7-8630-2de5a0b7f39b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.546006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 401/822] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:02.547094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f79f1a-f1a0-46f7-8630-2de5a0b7f39b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.563681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f295ca-d519-4a30-ad48-66a9079360c7 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.605194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be00e4-566f-471c-a5f8-dd79f8c6116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.606160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be00e4-566f-471c-a5f8-dd79f8c6116d None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.606453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be00e4-566f-471c-a5f8-dd79f8c6116d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.606760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be00e4-566f-471c-a5f8-dd79f8c6116d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.607516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be00e4-566f-471c-a5f8-dd79f8c6116d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.609344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be00e4-566f-471c-a5f8-dd79f8c6116d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.662433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f79f1a-f1a0-46f7-8630-2de5a0b7f39b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.698813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb4fb5-5405-4271-a0af-a8f46b2adde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.699485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb4fb5-5405-4271-a0af-a8f46b2adde6 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.699767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb4fb5-5405-4271-a0af-a8f46b2adde6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.700039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb4fb5-5405-4271-a0af-a8f46b2adde6 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.700731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb4fb5-5405-4271-a0af-a8f46b2adde6 None admin] RBAC: Adding query filter params (name=network) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:02.701030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb4fb5-5405-4271-a0af-a8f46b2adde6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.702377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb4fb5-5405-4271-a0af-a8f46b2adde6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.706912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 402/823] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:02.707656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb4fb5-5405-4271-a0af-a8f46b2adde6 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.716787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 422/824] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 05:41:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:02.720199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be00e4-566f-471c-a5f8-dd79f8c6116d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.746807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.830766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb4fb5-5405-4271-a0af-a8f46b2adde6 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.844889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be00e4-566f-471c-a5f8-dd79f8c6116d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.862110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.887842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c6d87-06f1-49fb-9ed8-418ea0f7c6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.888754 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c6d87-06f1-49fb-9ed8-418ea0f7c6b0 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.889295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c6d87-06f1-49fb-9ed8-418ea0f7c6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.889660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c6d87-06f1-49fb-9ed8-418ea0f7c6b0 None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/56d11b3ed0474760a10e6f94df838297/roles/9cc15ca9e61f425cacb9c1a9f52073dd` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.890446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14632b-ae40-45fb-995d-8beec61b411e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.891547 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14632b-ae40-45fb-995d-8beec61b411e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.892040 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14632b-ae40-45fb-995d-8beec61b411e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.892522 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14632b-ae40-45fb-995d-8beec61b411e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.911707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c6d87-06f1-49fb-9ed8-418ea0f7c6b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=56d11b3ed0474760a10e6f94df838297, role_id=9cc15ca9e61f425cacb9c1a9f52073dd)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.912172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c6d87-06f1-49fb-9ed8-418ea0f7c6b0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.913547 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14632b-ae40-45fb-995d-8beec61b411e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.916114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14632b-ae40-45fb-995d-8beec61b411e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5c14632b-ae40-45fb-995d-8beec61b411e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:02.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:02.937744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:02.938719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:02.938719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:02.938719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:02.938719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:02.942538 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 423/825] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:02.943875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14632b-ae40-45fb-995d-8beec61b411e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.017720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 403/826] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:02 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/56d11b3ed0474760a10e6f94df838297/roles/9cc15ca9e61f425cacb9c1a9f52073dd => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:03.027578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c6d87-06f1-49fb-9ed8-418ea0f7c6b0 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.055854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14632b-ae40-45fb-995d-8beec61b411e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.099341 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37225e0f-2075-4af8-88f7-3912de9d1f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.100063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37225e0f-2075-4af8-88f7-3912de9d1f0c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.100327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37225e0f-2075-4af8-88f7-3912de9d1f0c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.100601 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37225e0f-2075-4af8-88f7-3912de9d1f0c None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.101496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37225e0f-2075-4af8-88f7-3912de9d1f0c None admin] RBAC: Adding query filter params (type=network) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:03.101735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37225e0f-2075-4af8-88f7-3912de9d1f0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.103358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37225e0f-2075-4af8-88f7-3912de9d1f0c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.120435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 424/827] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:03.125772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37225e0f-2075-4af8-88f7-3912de9d1f0c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.147661 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.168617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c6d87-06f1-49fb-9ed8-418ea0f7c6b0 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.184279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.206012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60a8b0-1f23-4619-b56d-a43518584925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.206864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60a8b0-1f23-4619-b56d-a43518584925 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.207253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60a8b0-1f23-4619-b56d-a43518584925 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.207687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60a8b0-1f23-4619-b56d-a43518584925 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.208576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60a8b0-1f23-4619-b56d-a43518584925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:03.208978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60a8b0-1f23-4619-b56d-a43518584925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.218504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60a8b0-1f23-4619-b56d-a43518584925 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.223781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 404/828] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:03.235687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60a8b0-1f23-4619-b56d-a43518584925 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.282566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37225e0f-2075-4af8-88f7-3912de9d1f0c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.324351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd856ee-8747-49e5-9043-de5f0fe23fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.325503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd856ee-8747-49e5-9043-de5f0fe23fda None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.325785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd856ee-8747-49e5-9043-de5f0fe23fda None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.326060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd856ee-8747-49e5-9043-de5f0fe23fda None admin] PATH_INFO: `/v3/endpoints` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.326869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd856ee-8747-49e5-9043-de5f0fe23fda None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.328273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd856ee-8747-49e5-9043-de5f0fe23fda None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.395140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60a8b0-1f23-4619-b56d-a43518584925 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.433991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5272f77-110c-4163-a291-66826c0c0b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.435536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5272f77-110c-4163-a291-66826c0c0b5f None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.436044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5272f77-110c-4163-a291-66826c0c0b5f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.436795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5272f77-110c-4163-a291-66826c0c0b5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.447542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 425/829] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 05:41:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:41:03.449216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5272f77-110c-4163-a291-66826c0c0b5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.451114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5272f77-110c-4163-a291-66826c0c0b5f None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.455501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-b5272f77-110c-4163-a291-66826c0c0b5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:03.470319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:03.471332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:03.472458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:03.472458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:03.472458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:03.472458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:03.472458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:03.472458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:03.472458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:03.472458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:03.475409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 405/830] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:03.481810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd856ee-8747-49e5-9043-de5f0fe23fda None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.613622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.644665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd856ee-8747-49e5-9043-de5f0fe23fda None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.681618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bdb3a-7432-4f5a-a2b0-7d4bc16ef5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.682340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bdb3a-7432-4f5a-a2b0-7d4bc16ef5fb None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.682696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bdb3a-7432-4f5a-a2b0-7d4bc16ef5fb None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.682918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bdb3a-7432-4f5a-a2b0-7d4bc16ef5fb None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.683733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bdb3a-7432-4f5a-a2b0-7d4bc16ef5fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:03.684071 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bdb3a-7432-4f5a-a2b0-7d4bc16ef5fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.685468 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bdb3a-7432-4f5a-a2b0-7d4bc16ef5fb None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.698506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 426/831] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:03.704650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5272f77-110c-4163-a291-66826c0c0b5f None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.836043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5272f77-110c-4163-a291-66826c0c0b5f None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.875295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b729c-2215-4478-90c2-78d86736a314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.876118 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b729c-2215-4478-90c2-78d86736a314 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.876394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b729c-2215-4478-90c2-78d86736a314 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.876667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b729c-2215-4478-90c2-78d86736a314 None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/5c359fc0168141fb8da3f953884fd2e5/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.879556 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b729c-2215-4478-90c2-78d86736a314 None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=5c359fc0168141fb8da3f953884fd2e5, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.881065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b729c-2215-4478-90c2-78d86736a314 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.964149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 406/832] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:03 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/5c359fc0168141fb8da3f953884fd2e5/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:04.644775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d3458-275c-41b2-bd20-508c13e21e26 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.645351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d3458-275c-41b2-bd20-508c13e21e26 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.645351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d3458-275c-41b2-bd20-508c13e21e26 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.646991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 427/833] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:04.652217 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d70e40-5e7e-4ff5-a1c0-7fdabba71b35 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.652509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d70e40-5e7e-4ff5-a1c0-7fdabba71b35 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.652791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d70e40-5e7e-4ff5-a1c0-7fdabba71b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.670347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a8d70e40-5e7e-4ff5-a1c0-7fdabba71b35 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:04.671575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a8d70e40-5e7e-4ff5-a1c0-7fdabba71b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:04.736250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:04.749330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 407/834] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:04.771002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d3458-275c-41b2-bd20-508c13e21e26 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:04.878100 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d3458-275c-41b2-bd20-508c13e21e26 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:04.919235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10007a1e-570d-4b7f-8614-6b597198ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:04.920069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10007a1e-570d-4b7f-8614-6b597198ca2f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.920355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10007a1e-570d-4b7f-8614-6b597198ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.920632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10007a1e-570d-4b7f-8614-6b597198ca2f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.921301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10007a1e-570d-4b7f-8614-6b597198ca2f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:04.922720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10007a1e-570d-4b7f-8614-6b597198ca2f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-10007a1e-570d-4b7f-8614-6b597198ca2f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:04.935484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:04.936666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:04.937748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 428/835] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:04.942764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d70e40-5e7e-4ff5-a1c0-7fdabba71b35 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.009834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87d1eb-2e16-4270-97bf-776d2c4c5f9f None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.010145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87d1eb-2e16-4270-97bf-776d2c4c5f9f None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.010587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87d1eb-2e16-4270-97bf-776d2c4c5f9f None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.012292 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 429/836] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:05.017536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108487f-b2c3-4246-8775-af876cf0c249 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.017825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108487f-b2c3-4246-8775-af876cf0c249 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.018103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108487f-b2c3-4246-8775-af876cf0c249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.030781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.037149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9108487f-b2c3-4246-8775-af876cf0c249 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:05.038355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9108487f-b2c3-4246-8775-af876cf0c249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:05.049412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d70e40-5e7e-4ff5-a1c0-7fdabba71b35 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.085218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1b0e2-1f0d-4bca-921e-370436b62c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.086232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1b0e2-1f0d-4bca-921e-370436b62c85 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.086745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1b0e2-1f0d-4bca-921e-370436b62c85 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.087207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1b0e2-1f0d-4bca-921e-370436b62c85 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.088650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1b0e2-1f0d-4bca-921e-370436b62c85 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:05.089140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1b0e2-1f0d-4bca-921e-370436b62c85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.090728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1b0e2-1f0d-4bca-921e-370436b62c85 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.095990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 408/837] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:05.104473 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1b0e2-1f0d-4bca-921e-370436b62c85 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.120237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 430/838] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:05.142943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108487f-b2c3-4246-8775-af876cf0c249 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.175228 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.223160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1b0e2-1f0d-4bca-921e-370436b62c85 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.248934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108487f-b2c3-4246-8775-af876cf0c249 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.257980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ddcca-4544-4d0d-a16d-e3fefa8088ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.258984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ddcca-4544-4d0d-a16d-e3fefa8088ed None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.259453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ddcca-4544-4d0d-a16d-e3fefa8088ed None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.260480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ddcca-4544-4d0d-a16d-e3fefa8088ed None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.261368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ddcca-4544-4d0d-a16d-e3fefa8088ed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:05.261858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ddcca-4544-4d0d-a16d-e3fefa8088ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.263466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ddcca-4544-4d0d-a16d-e3fefa8088ed None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.268850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 409/839] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:05.295427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84701513-d98a-4b69-80a3-beefbc526b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.296253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84701513-d98a-4b69-80a3-beefbc526b31 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.296534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84701513-d98a-4b69-80a3-beefbc526b31 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.296808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84701513-d98a-4b69-80a3-beefbc526b31 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.297483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84701513-d98a-4b69-80a3-beefbc526b31 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.298938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84701513-d98a-4b69-80a3-beefbc526b31 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.302195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-84701513-d98a-4b69-80a3-beefbc526b31 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:05.313363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:05.325532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:05.326404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 431/840] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:05.327269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ddcca-4544-4d0d-a16d-e3fefa8088ed None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.435247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ddcca-4544-4d0d-a16d-e3fefa8088ed None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.456970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.475119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defca161-5a61-4bcc-8c2e-faa2f6922316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.475849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defca161-5a61-4bcc-8c2e-faa2f6922316 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.476134 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defca161-5a61-4bcc-8c2e-faa2f6922316 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.476413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defca161-5a61-4bcc-8c2e-faa2f6922316 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.477296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defca161-5a61-4bcc-8c2e-faa2f6922316 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:05.477664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defca161-5a61-4bcc-8c2e-faa2f6922316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.479101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defca161-5a61-4bcc-8c2e-faa2f6922316 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.483953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 410/841] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:05.489802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84701513-d98a-4b69-80a3-beefbc526b31 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.506296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74090657-a09e-43ec-b2b9-3ebd054abe22 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.506656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74090657-a09e-43ec-b2b9-3ebd054abe22 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.506851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74090657-a09e-43ec-b2b9-3ebd054abe22 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.508430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 411/842] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:05.514111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a0af9-ebca-40cb-96ff-3655d64717dd None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.514393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a0af9-ebca-40cb-96ff-3655d64717dd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.514702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a0af9-ebca-40cb-96ff-3655d64717dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.532029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a44a0af9-ebca-40cb-96ff-3655d64717dd None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:05.533235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a44a0af9-ebca-40cb-96ff-3655d64717dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:05.605781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84701513-d98a-4b69-80a3-beefbc526b31 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.623079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 412/843] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:05.645207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2139c26-0867-4a9c-8895-a013d1d34d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.646657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2139c26-0867-4a9c-8895-a013d1d34d34 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.647089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2139c26-0867-4a9c-8895-a013d1d34d34 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.647413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a0af9-ebca-40cb-96ff-3655d64717dd None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.647757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2139c26-0867-4a9c-8895-a013d1d34d34 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.648620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2139c26-0867-4a9c-8895-a013d1d34d34 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:05.649085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2139c26-0867-4a9c-8895-a013d1d34d34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.652494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2139c26-0867-4a9c-8895-a013d1d34d34 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.663020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 432/844] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:05.765070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a0af9-ebca-40cb-96ff-3655d64717dd None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.780367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.803149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c130524-b740-464d-84bb-d95e796bd7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.803968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130524-b740-464d-84bb-d95e796bd7c3 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.804247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130524-b740-464d-84bb-d95e796bd7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.804513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130524-b740-464d-84bb-d95e796bd7c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.817661 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c130524-b740-464d-84bb-d95e796bd7c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-6c130524-b740-464d-84bb-d95e796bd7c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:05.825887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:05.826959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:05.827825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 413/845] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:05.833508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2139c26-0867-4a9c-8895-a013d1d34d34 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.905085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.945212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2139c26-0867-4a9c-8895-a013d1d34d34 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.981379 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90139314-99ba-4a48-947d-1edd2ae8d2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.982048 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90139314-99ba-4a48-947d-1edd2ae8d2d8 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.982326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90139314-99ba-4a48-947d-1edd2ae8d2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.982636 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90139314-99ba-4a48-947d-1edd2ae8d2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.983387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90139314-99ba-4a48-947d-1edd2ae8d2d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:05.983692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90139314-99ba-4a48-947d-1edd2ae8d2d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.985062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90139314-99ba-4a48-947d-1edd2ae8d2d8 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.997822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 433/846] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.002971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c130524-b740-464d-84bb-d95e796bd7c3 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.103213 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb214caa-02f5-4460-a68b-982f4d697300 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.104023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb214caa-02f5-4460-a68b-982f4d697300 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.104850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb214caa-02f5-4460-a68b-982f4d697300 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.107141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 434/847] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:06.112308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c930251-ca5c-4ef4-8397-2b6ddb177b73 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.112813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c930251-ca5c-4ef4-8397-2b6ddb177b73 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.113340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c930251-ca5c-4ef4-8397-2b6ddb177b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.116555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c130524-b740-464d-84bb-d95e796bd7c3 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.130331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9c930251-ca5c-4ef4-8397-2b6ddb177b73 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:06.131843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9c930251-ca5c-4ef4-8397-2b6ddb177b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:06.152850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc3e3e-3185-42f0-abae-e0fcbf70c6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.153648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc3e3e-3185-42f0-abae-e0fcbf70c6e3 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.153928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc3e3e-3185-42f0-abae-e0fcbf70c6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.154200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc3e3e-3185-42f0-abae-e0fcbf70c6e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.165718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc3e3e-3185-42f0-abae-e0fcbf70c6e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.167211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc3e3e-3185-42f0-abae-e0fcbf70c6e3 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-0bbc3e3e-3185-42f0-abae-e0fcbf70c6e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:06.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:06.186764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:06.187697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:06.187697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:06.187697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:06.187697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:06.187697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 414/848] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:06.199456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc3e3e-3185-42f0-abae-e0fcbf70c6e3 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.218701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 435/849] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:06.231229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.235506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c930251-ca5c-4ef4-8397-2b6ddb177b73 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.296861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.311366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc3e3e-3185-42f0-abae-e0fcbf70c6e3 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.342349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c930251-ca5c-4ef4-8397-2b6ddb177b73 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.359085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67052f3-477c-40b8-8c8e-22e882815764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.360045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67052f3-477c-40b8-8c8e-22e882815764 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.360390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67052f3-477c-40b8-8c8e-22e882815764 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.360730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67052f3-477c-40b8-8c8e-22e882815764 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.361516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67052f3-477c-40b8-8c8e-22e882815764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:06.361888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67052f3-477c-40b8-8c8e-22e882815764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.364010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67052f3-477c-40b8-8c8e-22e882815764 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.376646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc25f00-6ee1-400e-8285-fc161a6a7c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.377436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc25f00-6ee1-400e-8285-fc161a6a7c1c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.377715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc25f00-6ee1-400e-8285-fc161a6a7c1c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.377989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc25f00-6ee1-400e-8285-fc161a6a7c1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.379065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 415/850] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.384205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67052f3-477c-40b8-8c8e-22e882815764 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.395217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc25f00-6ee1-400e-8285-fc161a6a7c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-7cc25f00-6ee1-400e-8285-fc161a6a7c1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:06.405761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:06.406955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:06.407823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:06.408470 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 436/851] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:06.415089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc25f00-6ee1-400e-8285-fc161a6a7c1c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.482613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.499295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67052f3-477c-40b8-8c8e-22e882815764 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.535997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f58196-4d1a-4df6-925d-22dc8a7b1aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.536808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f58196-4d1a-4df6-925d-22dc8a7b1aed None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.537142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f58196-4d1a-4df6-925d-22dc8a7b1aed None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.537412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f58196-4d1a-4df6-925d-22dc8a7b1aed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.545092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f58196-4d1a-4df6-925d-22dc8a7b1aed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.546547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f58196-4d1a-4df6-925d-22dc8a7b1aed None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.548922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc25f00-6ee1-400e-8285-fc161a6a7c1c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-84f58196-4d1a-4df6-925d-22dc8a7b1aed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:06.567077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:06.568196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:06.569141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:06.569141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:06.569141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:06.569141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:06.569141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:06.569141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:06.569141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:06.569141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:06.569141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:06.569141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:06.571203 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.573930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 416/852] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:06.574874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f58196-4d1a-4df6-925d-22dc8a7b1aed None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.591444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56f362-4c96-44b4-a9f9-3ef20fc3425e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.592487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56f362-4c96-44b4-a9f9-3ef20fc3425e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.593015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56f362-4c96-44b4-a9f9-3ef20fc3425e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.593544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56f362-4c96-44b4-a9f9-3ef20fc3425e None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.594632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56f362-4c96-44b4-a9f9-3ef20fc3425e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:06.595137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56f362-4c96-44b4-a9f9-3ef20fc3425e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.596739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56f362-4c96-44b4-a9f9-3ef20fc3425e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.609011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 437/853] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:06.614603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56f362-4c96-44b4-a9f9-3ef20fc3425e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.681022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f58196-4d1a-4df6-925d-22dc8a7b1aed None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.715840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7cb2e-eea9-4840-856e-3189eb6b2e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.716711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7cb2e-eea9-4840-856e-3189eb6b2e52 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.717138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7cb2e-eea9-4840-856e-3189eb6b2e52 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.717555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7cb2e-eea9-4840-856e-3189eb6b2e52 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.718642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7cb2e-eea9-4840-856e-3189eb6b2e52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:06.719136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7cb2e-eea9-4840-856e-3189eb6b2e52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.720436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56f362-4c96-44b4-a9f9-3ef20fc3425e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.721230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7cb2e-eea9-4840-856e-3189eb6b2e52 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.744135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 417/854] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.745895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7cb2e-eea9-4840-856e-3189eb6b2e52 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.748734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.758648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0d0b4-9e5b-46dc-9ef1-0b2d34e4b7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.759489 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0d0b4-9e5b-46dc-9ef1-0b2d34e4b7b5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.759723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0d0b4-9e5b-46dc-9ef1-0b2d34e4b7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.760101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0d0b4-9e5b-46dc-9ef1-0b2d34e4b7b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.771090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0d0b4-9e5b-46dc-9ef1-0b2d34e4b7b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.772555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0d0b4-9e5b-46dc-9ef1-0b2d34e4b7b5 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-70a0d0b4-9e5b-46dc-9ef1-0b2d34e4b7b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:06.790221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:06.791918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:06.792898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:06.792898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:06.792898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:06.792898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:06.796810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 438/855] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:06.797663 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0d0b4-9e5b-46dc-9ef1-0b2d34e4b7b5 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.830023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.862341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7cb2e-eea9-4840-856e-3189eb6b2e52 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.903426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea286d7-5864-4ceb-9efd-f3dd5d02fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.905613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0d0b4-9e5b-46dc-9ef1-0b2d34e4b7b5 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.907600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea286d7-5864-4ceb-9efd-f3dd5d02fc8a None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.909049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea286d7-5864-4ceb-9efd-f3dd5d02fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.912306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea286d7-5864-4ceb-9efd-f3dd5d02fc8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.944871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04d92d-5603-4ba9-85cc-eeb054488306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.945572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04d92d-5603-4ba9-85cc-eeb054488306 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.945825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04d92d-5603-4ba9-85cc-eeb054488306 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.946099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04d92d-5603-4ba9-85cc-eeb054488306 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.946869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04d92d-5603-4ba9-85cc-eeb054488306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:06.947179 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04d92d-5603-4ba9-85cc-eeb054488306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.948593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04d92d-5603-4ba9-85cc-eeb054488306 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.964721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 439/856] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:06.965641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04d92d-5603-4ba9-85cc-eeb054488306 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.007054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea286d7-5864-4ceb-9efd-f3dd5d02fc8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.009481 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea286d7-5864-4ceb-9efd-f3dd5d02fc8a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.012091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.067038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04d92d-5603-4ba9-85cc-eeb054488306 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.069493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.111080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c05e27-d778-401c-8b85-8e2b4689ccb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.111882 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c05e27-d778-401c-8b85-8e2b4689ccb5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.112163 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c05e27-d778-401c-8b85-8e2b4689ccb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.112439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c05e27-d778-401c-8b85-8e2b4689ccb5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.122304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c05e27-d778-401c-8b85-8e2b4689ccb5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.123983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c05e27-d778-401c-8b85-8e2b4689ccb5 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-e9c05e27-d778-401c-8b85-8e2b4689ccb5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:07.142095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:07.143112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:07.144072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:07.144072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:07.144072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:07.144072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:07.144072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:07.144072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:07.144072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:07.144072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:07.144072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:07.144072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:07.145156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 418/858] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:07.145421 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 440/858] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:07.146248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c05e27-d778-401c-8b85-8e2b4689ccb5 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.146920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea286d7-5864-4ceb-9efd-f3dd5d02fc8a None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.250691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c05e27-d778-401c-8b85-8e2b4689ccb5 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.266330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea286d7-5864-4ceb-9efd-f3dd5d02fc8a None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.268156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.286033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0337ec-dcde-41af-9b55-3eb4e423812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.286743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0337ec-dcde-41af-9b55-3eb4e423812e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.287026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0337ec-dcde-41af-9b55-3eb4e423812e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.287308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0337ec-dcde-41af-9b55-3eb4e423812e None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.288154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0337ec-dcde-41af-9b55-3eb4e423812e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:07.288465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0337ec-dcde-41af-9b55-3eb4e423812e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.289902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0337ec-dcde-41af-9b55-3eb4e423812e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.305147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92f3b7-dd00-421d-ba70-93e637ee0dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.306270 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92f3b7-dd00-421d-ba70-93e637ee0dd6 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.307318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92f3b7-dd00-421d-ba70-93e637ee0dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.307793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92f3b7-dd00-421d-ba70-93e637ee0dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.313980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 441/859] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:07.314972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0337ec-dcde-41af-9b55-3eb4e423812e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.319733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92f3b7-dd00-421d-ba70-93e637ee0dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.321411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92f3b7-dd00-421d-ba70-93e637ee0dd6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.326347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-9f92f3b7-dd00-421d-ba70-93e637ee0dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:07.341383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:07.344937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:07.345924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:07.345924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:07.345924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:07.345924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:07.345924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 419/860] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:07.348452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92f3b7-dd00-421d-ba70-93e637ee0dd6 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.421573 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0337ec-dcde-41af-9b55-3eb4e423812e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.456387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92f3b7-dd00-421d-ba70-93e637ee0dd6 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.456912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa995df-5990-43d3-b046-af0cc754a6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.457563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa995df-5990-43d3-b046-af0cc754a6b6 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.458014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa995df-5990-43d3-b046-af0cc754a6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.458295 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa995df-5990-43d3-b046-af0cc754a6b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.492460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05966d04-6058-4b62-bf30-5207423471e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.493130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05966d04-6058-4b62-bf30-5207423471e3 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.493430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05966d04-6058-4b62-bf30-5207423471e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.493693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05966d04-6058-4b62-bf30-5207423471e3 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.494525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05966d04-6058-4b62-bf30-5207423471e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:07.494827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05966d04-6058-4b62-bf30-5207423471e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.496203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05966d04-6058-4b62-bf30-5207423471e3 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.514421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 420/861] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:07.514421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acd45c-9369-48a4-820a-b9222d430899 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.514421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acd45c-9369-48a4-820a-b9222d430899 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.514421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acd45c-9369-48a4-820a-b9222d430899 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.514937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 421/862] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:07.515327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acd45c-9369-48a4-820a-b9222d430899 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.535677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.542383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa995df-5990-43d3-b046-af0cc754a6b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.543899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa995df-5990-43d3-b046-af0cc754a6b6 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.554414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.630247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acd45c-9369-48a4-820a-b9222d430899 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.655673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 442/863] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:07.655673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5581437-5cfd-4815-bad5-b1c506515a10 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.655673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5581437-5cfd-4815-bad5-b1c506515a10 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.655673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5581437-5cfd-4815-bad5-b1c506515a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.669810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d5581437-5cfd-4815-bad5-b1c506515a10 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:07.671067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842583d-d5a5-4fa1-9f9b-e69be2f9d6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.671193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d5581437-5cfd-4815-bad5-b1c506515a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:07.671954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842583d-d5a5-4fa1-9f9b-e69be2f9d6a5 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.672308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842583d-d5a5-4fa1-9f9b-e69be2f9d6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.672656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842583d-d5a5-4fa1-9f9b-e69be2f9d6a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.680043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842583d-d5a5-4fa1-9f9b-e69be2f9d6a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.681600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842583d-d5a5-4fa1-9f9b-e69be2f9d6a5 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-1842583d-d5a5-4fa1-9f9b-e69be2f9d6a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:07.701512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:07.702687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:07.703830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:07.703830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:07.703830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:07.703830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:07.703830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:07.703830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:07.703830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:07.703830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:07.709866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 422/864] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:07.709866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842583d-d5a5-4fa1-9f9b-e69be2f9d6a5 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.778852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 443/865] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:07.779642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5581437-5cfd-4815-bad5-b1c506515a10 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.801467 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.840977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.845053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842583d-d5a5-4fa1-9f9b-e69be2f9d6a5 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.887417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ae9b1-a450-4c6c-ab96-fbdd606779d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.888216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ae9b1-a450-4c6c-ab96-fbdd606779d8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.888497 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ae9b1-a450-4c6c-ab96-fbdd606779d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.889016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ae9b1-a450-4c6c-ab96-fbdd606779d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.900628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ae9b1-a450-4c6c-ab96-fbdd606779d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.902067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ae9b1-a450-4c6c-ab96-fbdd606779d8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.910765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5581437-5cfd-4815-bad5-b1c506515a10 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-079ae9b1-a450-4c6c-ab96-fbdd606779d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:07.921810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:07.922834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:07.923752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:07.923752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:07.923752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:07.923752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:07.925737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 423/866] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:07.925737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ae9b1-a450-4c6c-ab96-fbdd606779d8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.953266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcbd2c-ecb1-4850-9bca-4d3b266d2b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.953939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcbd2c-ecb1-4850-9bca-4d3b266d2b3e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.954385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcbd2c-ecb1-4850-9bca-4d3b266d2b3e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.954692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcbd2c-ecb1-4850-9bca-4d3b266d2b3e None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.955485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcbd2c-ecb1-4850-9bca-4d3b266d2b3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:07.955792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcbd2c-ecb1-4850-9bca-4d3b266d2b3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.957212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcbd2c-ecb1-4850-9bca-4d3b266d2b3e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.969216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 444/867] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:07.970645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d9adc-acfd-4743-9555-84bfb96a3381 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.971006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d9adc-acfd-4743-9555-84bfb96a3381 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.971356 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d9adc-acfd-4743-9555-84bfb96a3381 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.972951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 445/868] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:07.973724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d9adc-acfd-4743-9555-84bfb96a3381 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.038880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ae9b1-a450-4c6c-ab96-fbdd606779d8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.064618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:08.073407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2da571-0b6f-4358-83ee-f06e048c4ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.074066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2da571-0b6f-4358-83ee-f06e048c4ab9 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.074343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2da571-0b6f-4358-83ee-f06e048c4ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.077712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2da571-0b6f-4358-83ee-f06e048c4ab9 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.077712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2da571-0b6f-4358-83ee-f06e048c4ab9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.077712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2da571-0b6f-4358-83ee-f06e048c4ab9 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.077981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:08.081705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d9adc-acfd-4743-9555-84bfb96a3381 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.114913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fb73f-6132-441b-a5d2-e2d495cb6b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.115565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fb73f-6132-441b-a5d2-e2d495cb6b16 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.115879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fb73f-6132-441b-a5d2-e2d495cb6b16 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.116155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fb73f-6132-441b-a5d2-e2d495cb6b16 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.116875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fb73f-6132-441b-a5d2-e2d495cb6b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:08.117181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fb73f-6132-441b-a5d2-e2d495cb6b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.118667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fb73f-6132-441b-a5d2-e2d495cb6b16 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.128562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 424/869] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:41:07 2022] POST /identity/v3/services => generated 269 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:08.129806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25b118-a426-4f50-b2b6-d2ff8615f654 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.134728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25b118-a426-4f50-b2b6-d2ff8615f654 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.135545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 446/870] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:08.135688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25b118-a426-4f50-b2b6-d2ff8615f654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.135800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fb73f-6132-441b-a5d2-e2d495cb6b16 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.152087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ba25b118-a426-4f50-b2b6-d2ff8615f654 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:08.153316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ba25b118-a426-4f50-b2b6-d2ff8615f654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:08.242502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 425/871] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:08 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:08.242502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25b118-a426-4f50-b2b6-d2ff8615f654 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.270762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fb73f-6132-441b-a5d2-e2d495cb6b16 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.309107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcea71-b454-4a52-81eb-4b74a610451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.309696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcea71-b454-4a52-81eb-4b74a610451c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.309984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcea71-b454-4a52-81eb-4b74a610451c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.310264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcea71-b454-4a52-81eb-4b74a610451c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.311196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcea71-b454-4a52-81eb-4b74a610451c None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=56d11b3ed0474760a10e6f94df838297) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:08.311530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcea71-b454-4a52-81eb-4b74a610451c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.312958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcea71-b454-4a52-81eb-4b74a610451c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.319207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 447/872] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:08 2022] GET /identity/v3/role_assignments?user.id=56d11b3ed0474760a10e6f94df838297&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:08.319985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcea71-b454-4a52-81eb-4b74a610451c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.352119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:08.358500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25b118-a426-4f50-b2b6-d2ff8615f654 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.373432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:08.398878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e326428-b97c-44e4-b464-1b3b3b7a90e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.399737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e326428-b97c-44e4-b464-1b3b3b7a90e6 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.400023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e326428-b97c-44e4-b464-1b3b3b7a90e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.400295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e326428-b97c-44e4-b464-1b3b3b7a90e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.408583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e326428-b97c-44e4-b464-1b3b3b7a90e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.410061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e326428-b97c-44e4-b464-1b3b3b7a90e6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-3e326428-b97c-44e4-b464-1b3b3b7a90e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:08.426089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:08.427109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:08.428037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:08.428037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:08.428037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:08.428037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:08.428037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:08.428037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:08.428037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:08.428037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:08.428037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 426/873] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:08.433459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e326428-b97c-44e4-b464-1b3b3b7a90e6 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.442320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcea71-b454-4a52-81eb-4b74a610451c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.480698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1d8f8-522d-4fae-9dfa-7d9dfa5a2cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.481701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1d8f8-522d-4fae-9dfa-7d9dfa5a2cdf None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.482083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1d8f8-522d-4fae-9dfa-7d9dfa5a2cdf None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.482456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1d8f8-522d-4fae-9dfa-7d9dfa5a2cdf None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.483398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1d8f8-522d-4fae-9dfa-7d9dfa5a2cdf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.484943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1d8f8-522d-4fae-9dfa-7d9dfa5a2cdf None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.545283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e326428-b97c-44e4-b464-1b3b3b7a90e6 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.570229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 448/874] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:41:08 2022] POST /identity/v3/services => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:08.592153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363b607-d9f0-4275-8bf2-b39eba54bc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.592842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363b607-d9f0-4275-8bf2-b39eba54bc4d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.593124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363b607-d9f0-4275-8bf2-b39eba54bc4d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.594452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363b607-d9f0-4275-8bf2-b39eba54bc4d None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.594452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363b607-d9f0-4275-8bf2-b39eba54bc4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:08.594724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363b607-d9f0-4275-8bf2-b39eba54bc4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.596002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363b607-d9f0-4275-8bf2-b39eba54bc4d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.612905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 427/875] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:08.620947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1d8f8-522d-4fae-9dfa-7d9dfa5a2cdf None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.725146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:08.744734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1d8f8-522d-4fae-9dfa-7d9dfa5a2cdf None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.780036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957df6f-29af-4adc-8422-d77db3b1df28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.780795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957df6f-29af-4adc-8422-d77db3b1df28 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.780965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957df6f-29af-4adc-8422-d77db3b1df28 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.781239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957df6f-29af-4adc-8422-d77db3b1df28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.782070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957df6f-29af-4adc-8422-d77db3b1df28 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=5c359fc0168141fb8da3f953884fd2e5) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:08.782378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957df6f-29af-4adc-8422-d77db3b1df28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.783844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957df6f-29af-4adc-8422-d77db3b1df28 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.791078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 449/876] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:08 2022] GET /identity/v3/role_assignments?user.id=5c359fc0168141fb8da3f953884fd2e5&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=7a833c45603a4830b888e58e1232712a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:10.224546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433de6a3-838f-4958-8888-a4a393c9c3bc None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.225043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433de6a3-838f-4958-8888-a4a393c9c3bc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.225043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433de6a3-838f-4958-8888-a4a393c9c3bc None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.226688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 428/877] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:10.232062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033c070-192b-41c8-96c4-d419447ea584 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.232459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033c070-192b-41c8-96c4-d419447ea584 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.232650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033c070-192b-41c8-96c4-d419447ea584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.250084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-0033c070-192b-41c8-96c4-d419447ea584 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:10.251320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0033c070-192b-41c8-96c4-d419447ea584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:10.320987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 450/878] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:10.343754 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433de6a3-838f-4958-8888-a4a393c9c3bc None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.374398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.453827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433de6a3-838f-4958-8888-a4a393c9c3bc None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.489787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d454c8-b795-4217-8738-ac2290e29d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.490542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d454c8-b795-4217-8738-ac2290e29d34 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.490825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d454c8-b795-4217-8738-ac2290e29d34 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.491099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d454c8-b795-4217-8738-ac2290e29d34 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.491753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d454c8-b795-4217-8738-ac2290e29d34 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.493122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d454c8-b795-4217-8738-ac2290e29d34 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-a7d454c8-b795-4217-8738-ac2290e29d34 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:10.506407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:10.507391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:10.508663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 429/879] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:10.514155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033c070-192b-41c8-96c4-d419447ea584 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.580447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96951f32-93df-4ac6-9f86-89d19662ed72 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.580736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96951f32-93df-4ac6-9f86-89d19662ed72 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.581003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96951f32-93df-4ac6-9f86-89d19662ed72 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.586308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 430/880] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:10.587731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba2ef-f8f6-49bf-ab44-b903125626bb None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.588571 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba2ef-f8f6-49bf-ab44-b903125626bb None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.588847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba2ef-f8f6-49bf-ab44-b903125626bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.595398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.607685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-fe4ba2ef-f8f6-49bf-ab44-b903125626bb None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:10.609486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-fe4ba2ef-f8f6-49bf-ab44-b903125626bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:10.618687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033c070-192b-41c8-96c4-d419447ea584 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.661949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26876810-21c0-4edc-adc5-0c915c731926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.662782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26876810-21c0-4edc-adc5-0c915c731926 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.663158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26876810-21c0-4edc-adc5-0c915c731926 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.663502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26876810-21c0-4edc-adc5-0c915c731926 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.664297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26876810-21c0-4edc-adc5-0c915c731926 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:10.664671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26876810-21c0-4edc-adc5-0c915c731926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.666219 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26876810-21c0-4edc-adc5-0c915c731926 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:10.675832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 451/881] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:10.677754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26876810-21c0-4edc-adc5-0c915c731926 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.706320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 431/882] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:10.711079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe29ed-f269-4d31-9413-cc23c9491919 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.711694 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe29ed-f269-4d31-9413-cc23c9491919 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.711984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe29ed-f269-4d31-9413-cc23c9491919 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.714560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 432/883] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:10.719716 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8090fc1-dcea-4133-9788-b3ac6c49aed7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.720006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8090fc1-dcea-4133-9788-b3ac6c49aed7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.720284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8090fc1-dcea-4133-9788-b3ac6c49aed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.737254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a8090fc1-dcea-4133-9788-b3ac6c49aed7 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:10.738485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a8090fc1-dcea-4133-9788-b3ac6c49aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:10.756267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.800008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26876810-21c0-4edc-adc5-0c915c731926 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.808653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 433/884] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:10.809491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8090fc1-dcea-4133-9788-b3ac6c49aed7 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.842786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22397183-7437-4c9b-99cb-98f3cfb6f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.843572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397183-7437-4c9b-99cb-98f3cfb6f83a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.843915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397183-7437-4c9b-99cb-98f3cfb6f83a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.844262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397183-7437-4c9b-99cb-98f3cfb6f83a None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.844976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22397183-7437-4c9b-99cb-98f3cfb6f83a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:10.845290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22397183-7437-4c9b-99cb-98f3cfb6f83a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.846684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22397183-7437-4c9b-99cb-98f3cfb6f83a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:10.854823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 452/885] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:10.854823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22397183-7437-4c9b-99cb-98f3cfb6f83a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.859085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.918520 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8090fc1-dcea-4133-9788-b3ac6c49aed7 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.953011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875266-75fa-4577-af44-9dabc6697d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.953812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875266-75fa-4577-af44-9dabc6697d72 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.954092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875266-75fa-4577-af44-9dabc6697d72 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.954365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875266-75fa-4577-af44-9dabc6697d72 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.956013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22397183-7437-4c9b-99cb-98f3cfb6f83a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.964757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03875266-75fa-4577-af44-9dabc6697d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-03875266-75fa-4577-af44-9dabc6697d72 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:10.972907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:10.973857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:10.974759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 434/886] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:10.975671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875266-75fa-4577-af44-9dabc6697d72 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.989278 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.989714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66808d3f-cf7e-4a03-bb03-4a1fb1f97425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.990548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66808d3f-cf7e-4a03-bb03-4a1fb1f97425 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.990845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66808d3f-cf7e-4a03-bb03-4a1fb1f97425 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.991141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66808d3f-cf7e-4a03-bb03-4a1fb1f97425 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.991780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66808d3f-cf7e-4a03-bb03-4a1fb1f97425 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.993160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66808d3f-cf7e-4a03-bb03-4a1fb1f97425 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-66808d3f-cf7e-4a03-bb03-4a1fb1f97425 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:11.005272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:11.007706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:11.008692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 453/887] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:11.009509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0458ddd-05e1-4f04-b4cc-d22409b5be42 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.009509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0458ddd-05e1-4f04-b4cc-d22409b5be42 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.009509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0458ddd-05e1-4f04-b4cc-d22409b5be42 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.009509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 454/888] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:11.010708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0458ddd-05e1-4f04-b4cc-d22409b5be42 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.076528 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.077640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875266-75fa-4577-af44-9dabc6697d72 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.109155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0458ddd-05e1-4f04-b4cc-d22409b5be42 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.110675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ee971-06e6-4359-b94e-ef72b92dab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.111320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ee971-06e6-4359-b94e-ef72b92dab19 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.111604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ee971-06e6-4359-b94e-ef72b92dab19 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.111888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ee971-06e6-4359-b94e-ef72b92dab19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.112671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ee971-06e6-4359-b94e-ef72b92dab19 None admin] RBAC: Adding query filter params (interface=public, service_id=bed252d9aabb4d008cc6637efc3cf19a, region_id=RegionOne) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:11.112984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ee971-06e6-4359-b94e-ef72b92dab19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.114352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ee971-06e6-4359-b94e-ef72b92dab19 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.119871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 435/889] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/endpoints?service_id=bed252d9aabb4d008cc6637efc3cf19a&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:11.120657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ee971-06e6-4359-b94e-ef72b92dab19 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.144891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140bef8-c38e-4c39-88c9-603120df6f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.145635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140bef8-c38e-4c39-88c9-603120df6f45 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.145976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140bef8-c38e-4c39-88c9-603120df6f45 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.146257 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140bef8-c38e-4c39-88c9-603120df6f45 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.147090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140bef8-c38e-4c39-88c9-603120df6f45 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:11.147401 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140bef8-c38e-4c39-88c9-603120df6f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.148760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140bef8-c38e-4c39-88c9-603120df6f45 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.156488 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 455/890] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:11.157644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7cebb-fa2c-46b5-9c97-cee74d4d3d10 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.157925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7cebb-fa2c-46b5-9c97-cee74d4d3d10 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.158194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7cebb-fa2c-46b5-9c97-cee74d4d3d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.174995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4de7cebb-fa2c-46b5-9c97-cee74d4d3d10 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:11.176188 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4de7cebb-fa2c-46b5-9c97-cee74d4d3d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:11.197312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.219181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ee971-06e6-4359-b94e-ef72b92dab19 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.241934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 456/891] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:11.242694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7cebb-fa2c-46b5-9c97-cee74d4d3d10 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.271808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d50a0d-1d39-4858-9ef9-5dcbd42b8a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.272412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d50a0d-1d39-4858-9ef9-5dcbd42b8a3e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.272724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d50a0d-1d39-4858-9ef9-5dcbd42b8a3e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.273001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d50a0d-1d39-4858-9ef9-5dcbd42b8a3e None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.273701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d50a0d-1d39-4858-9ef9-5dcbd42b8a3e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:11.274008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d50a0d-1d39-4858-9ef9-5dcbd42b8a3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.275482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d50a0d-1d39-4858-9ef9-5dcbd42b8a3e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.280043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 436/892] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:11.280874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d50a0d-1d39-4858-9ef9-5dcbd42b8a3e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.299352 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.339245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7cebb-fa2c-46b5-9c97-cee74d4d3d10 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.372903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9ab19-9d77-4121-bd53-af0890714cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.373568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9ab19-9d77-4121-bd53-af0890714cbc None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.373848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9ab19-9d77-4121-bd53-af0890714cbc None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.374128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9ab19-9d77-4121-bd53-af0890714cbc None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.374843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9ab19-9d77-4121-bd53-af0890714cbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.376217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9ab19-9d77-4121-bd53-af0890714cbc None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.379558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d50a0d-1d39-4858-9ef9-5dcbd42b8a3e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.381750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 457/893] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/services => generated 1268 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:41:11.382537 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9ab19-9d77-4121-bd53-af0890714cbc None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.415733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f34a2-e721-43d3-b3cc-495bd2d37576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.417052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f34a2-e721-43d3-b3cc-495bd2d37576 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.417328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f34a2-e721-43d3-b3cc-495bd2d37576 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.417600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f34a2-e721-43d3-b3cc-495bd2d37576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.426530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.429801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f34a2-e721-43d3-b3cc-495bd2d37576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.431496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f34a2-e721-43d3-b3cc-495bd2d37576 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-8a4f34a2-e721-43d3-b3cc-495bd2d37576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:11.450997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:11.452044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:11.452970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:11.452970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:11.452970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:11.452970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:11.452970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 437/894] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:11.453255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f34a2-e721-43d3-b3cc-495bd2d37576 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.507683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9ab19-9d77-4121-bd53-af0890714cbc None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.544533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.552926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0823ba1-d7fe-454b-b586-a9fafebda7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.553933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0823ba1-d7fe-454b-b586-a9fafebda7e7 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.554042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0823ba1-d7fe-454b-b586-a9fafebda7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.554274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0823ba1-d7fe-454b-b586-a9fafebda7e7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.555256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0823ba1-d7fe-454b-b586-a9fafebda7e7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.556794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0823ba1-d7fe-454b-b586-a9fafebda7e7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.565989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f34a2-e721-43d3-b3cc-495bd2d37576 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-e0823ba1-d7fe-454b-b586-a9fafebda7e7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:11.570155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:11.571587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:11.572487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:11.575398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 458/895] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:11.576872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0823ba1-d7fe-454b-b586-a9fafebda7e7 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.603391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07791eb2-224d-4a5f-a999-00c0c487b3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.604612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07791eb2-224d-4a5f-a999-00c0c487b3bb None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.604881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07791eb2-224d-4a5f-a999-00c0c487b3bb None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.605153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07791eb2-224d-4a5f-a999-00c0c487b3bb None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.605851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07791eb2-224d-4a5f-a999-00c0c487b3bb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:11.606153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07791eb2-224d-4a5f-a999-00c0c487b3bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.607542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07791eb2-224d-4a5f-a999-00c0c487b3bb None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.612736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 438/896] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:11.613490 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07791eb2-224d-4a5f-a999-00c0c487b3bb None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.671179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.691173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0823ba1-d7fe-454b-b586-a9fafebda7e7 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.714610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07791eb2-224d-4a5f-a999-00c0c487b3bb None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.729196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6c74f-d344-496d-957f-d917b011bd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.729892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6c74f-d344-496d-957f-d917b011bd16 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.730162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6c74f-d344-496d-957f-d917b011bd16 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.730449 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6c74f-d344-496d-957f-d917b011bd16 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.731205 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6c74f-d344-496d-957f-d917b011bd16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:11.731513 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6c74f-d344-496d-957f-d917b011bd16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.732889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6c74f-d344-496d-957f-d917b011bd16 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.750103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 459/897] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:11.750103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6c74f-d344-496d-957f-d917b011bd16 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.754575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944abd2-f7e1-41c5-8d04-9394c064c8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.755252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944abd2-f7e1-41c5-8d04-9394c064c8a0 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.755532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944abd2-f7e1-41c5-8d04-9394c064c8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.755802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944abd2-f7e1-41c5-8d04-9394c064c8a0 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.756503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944abd2-f7e1-41c5-8d04-9394c064c8a0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:11.756804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944abd2-f7e1-41c5-8d04-9394c064c8a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.758156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944abd2-f7e1-41c5-8d04-9394c064c8a0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.763145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 439/898] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:11.763766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944abd2-f7e1-41c5-8d04-9394c064c8a0 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.802011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.854398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6c74f-d344-496d-957f-d917b011bd16 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.878061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944abd2-f7e1-41c5-8d04-9394c064c8a0 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.891670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ab498-f317-4ad4-9cde-062f87bc027d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.892548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ab498-f317-4ad4-9cde-062f87bc027d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.892969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ab498-f317-4ad4-9cde-062f87bc027d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.893421 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ab498-f317-4ad4-9cde-062f87bc027d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.894510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ab498-f317-4ad4-9cde-062f87bc027d None admin] RBAC: Adding query filter params (interface=public, service_id=a95eb4a84a034fefae9bb8cb474d5859, region_id=RegionOne) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:11.894975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ab498-f317-4ad4-9cde-062f87bc027d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.896505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ab498-f317-4ad4-9cde-062f87bc027d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.902417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 460/899] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/endpoints?service_id=a95eb4a84a034fefae9bb8cb474d5859&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:11.903403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ab498-f317-4ad4-9cde-062f87bc027d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.911920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.925200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707c1a-3055-421a-b9c5-aca9f319f548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.926028 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707c1a-3055-421a-b9c5-aca9f319f548 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.926311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707c1a-3055-421a-b9c5-aca9f319f548 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.927038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e707c1a-3055-421a-b9c5-aca9f319f548 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.934317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707c1a-3055-421a-b9c5-aca9f319f548 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.935792 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e707c1a-3055-421a-b9c5-aca9f319f548 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-9e707c1a-3055-421a-b9c5-aca9f319f548 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:11.951684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:11.952685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 440/900] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:11.953645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707c1a-3055-421a-b9c5-aca9f319f548 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.013499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ab498-f317-4ad4-9cde-062f87bc027d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.027749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:12.048809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efae43-2bb7-4263-a106-4d6aea9e50e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.049478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efae43-2bb7-4263-a106-4d6aea9e50e0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.049767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efae43-2bb7-4263-a106-4d6aea9e50e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.050033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efae43-2bb7-4263-a106-4d6aea9e50e0 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.050927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efae43-2bb7-4263-a106-4d6aea9e50e0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:12.051242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efae43-2bb7-4263-a106-4d6aea9e50e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.052657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efae43-2bb7-4263-a106-4d6aea9e50e0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.057324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 461/901] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:12.058121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efae43-2bb7-4263-a106-4d6aea9e50e0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.065996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e707c1a-3055-421a-b9c5-aca9f319f548 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.125384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7f34c-b6dc-44d1-bcd3-173884344084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.127138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7f34c-b6dc-44d1-bcd3-173884344084 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.127665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7f34c-b6dc-44d1-bcd3-173884344084 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.128348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7f34c-b6dc-44d1-bcd3-173884344084 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.129233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7f34c-b6dc-44d1-bcd3-173884344084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.131677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7f34c-b6dc-44d1-bcd3-173884344084 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.141316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 441/902] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/services => generated 1268 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:41:12.184409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efae43-2bb7-4263-a106-4d6aea9e50e0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.225582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f35c5-2b2c-4847-ae0d-d13368c5300c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.226376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f35c5-2b2c-4847-ae0d-d13368c5300c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.226696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f35c5-2b2c-4847-ae0d-d13368c5300c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.226990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f35c5-2b2c-4847-ae0d-d13368c5300c None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.227891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f35c5-2b2c-4847-ae0d-d13368c5300c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:12.228204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f35c5-2b2c-4847-ae0d-d13368c5300c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.229865 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f35c5-2b2c-4847-ae0d-d13368c5300c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.251702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 462/903] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:12.258054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7f34c-b6dc-44d1-bcd3-173884344084 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.311349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:12.368681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7f34c-b6dc-44d1-bcd3-173884344084 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.405074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badd8b0-f611-4022-874d-618dc9345277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.405960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badd8b0-f611-4022-874d-618dc9345277 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.406390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badd8b0-f611-4022-874d-618dc9345277 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.406849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badd8b0-f611-4022-874d-618dc9345277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.481584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badd8b0-f611-4022-874d-618dc9345277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.483068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badd8b0-f611-4022-874d-618dc9345277 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.533159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:12.592684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 442/904] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:12.598825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f35c5-2b2c-4847-ae0d-d13368c5300c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.634143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:12.709651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f35c5-2b2c-4847-ae0d-d13368c5300c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.746520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b4fa3-da94-4339-9904-51f4f428256b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.747342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b4fa3-da94-4339-9904-51f4f428256b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.747626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b4fa3-da94-4339-9904-51f4f428256b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.747908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b4fa3-da94-4339-9904-51f4f428256b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.759139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b4fa3-da94-4339-9904-51f4f428256b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.760636 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b4fa3-da94-4339-9904-51f4f428256b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-e56b4fa3-da94-4339-9904-51f4f428256b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:12.778956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:12.780085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:12.780977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:12.780977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:12.780977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:12.780977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:12.780977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 463/905] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:12.787277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badd8b0-f611-4022-874d-618dc9345277 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.895788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badd8b0-f611-4022-874d-618dc9345277 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.932096 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fad24-4db7-4aad-9886-82a4bd82c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.932782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fad24-4db7-4aad-9886-82a4bd82c567 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.933065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fad24-4db7-4aad-9886-82a4bd82c567 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.933339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fad24-4db7-4aad-9886-82a4bd82c567 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.934117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fad24-4db7-4aad-9886-82a4bd82c567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:12.934426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fad24-4db7-4aad-9886-82a4bd82c567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.935969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fad24-4db7-4aad-9886-82a4bd82c567 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.947964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 443/906] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:12.953294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b4fa3-da94-4339-9904-51f4f428256b None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.039830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.055127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b4fa3-da94-4339-9904-51f4f428256b None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.091234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa2dfb-8b51-47a5-8589-1bdefeab7c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.092030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa2dfb-8b51-47a5-8589-1bdefeab7c12 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.092311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa2dfb-8b51-47a5-8589-1bdefeab7c12 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.092584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa2dfb-8b51-47a5-8589-1bdefeab7c12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.099367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa2dfb-8b51-47a5-8589-1bdefeab7c12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.100916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa2dfb-8b51-47a5-8589-1bdefeab7c12 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-96fa2dfb-8b51-47a5-8589-1bdefeab7c12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:13.119115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:13.120052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:13.120961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:13.120961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:13.120961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:13.120961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:13.120961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:13.120961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:13.120961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:13.120961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:13.123501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 464/907] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:13.129052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fad24-4db7-4aad-9886-82a4bd82c567 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.133008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.233193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fad24-4db7-4aad-9886-82a4bd82c567 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.269556 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efca3cc-f10b-4b7e-b9e0-756c65ad7345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.270227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efca3cc-f10b-4b7e-b9e0-756c65ad7345 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.270533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efca3cc-f10b-4b7e-b9e0-756c65ad7345 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.270813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efca3cc-f10b-4b7e-b9e0-756c65ad7345 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.271603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efca3cc-f10b-4b7e-b9e0-756c65ad7345 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:13.271912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efca3cc-f10b-4b7e-b9e0-756c65ad7345 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.273337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efca3cc-f10b-4b7e-b9e0-756c65ad7345 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.286863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 444/908] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:13.293117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa2dfb-8b51-47a5-8589-1bdefeab7c12 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.403934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa2dfb-8b51-47a5-8589-1bdefeab7c12 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.436105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f16508-ea81-4105-b44f-24302ee29c9e None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.436613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f16508-ea81-4105-b44f-24302ee29c9e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.436692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f16508-ea81-4105-b44f-24302ee29c9e None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.438142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 445/909] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:13.441575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9c343-61b9-43f5-9526-823baa7f7377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.442308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9c343-61b9-43f5-9526-823baa7f7377 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.442628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9c343-61b9-43f5-9526-823baa7f7377 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.443031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9c343-61b9-43f5-9526-823baa7f7377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.443744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945a7ad-f317-4ec5-85d3-44c31bc483f5 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.443918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9c343-61b9-43f5-9526-823baa7f7377 None admin] RBAC: Adding query filter params (role.id=333abb19b7ef46c7a17f4f86c8d798e3, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=56d11b3ed0474760a10e6f94df838297) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:13.444031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945a7ad-f317-4ec5-85d3-44c31bc483f5 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.444182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9c343-61b9-43f5-9526-823baa7f7377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.444343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945a7ad-f317-4ec5-85d3-44c31bc483f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.445617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9c343-61b9-43f5-9526-823baa7f7377 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.453375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 465/910] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/role_assignments?user.id=56d11b3ed0474760a10e6f94df838297&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=333abb19b7ef46c7a17f4f86c8d798e3 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:13.465640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3945a7ad-f317-4ec5-85d3-44c31bc483f5 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:13.467359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3945a7ad-f317-4ec5-85d3-44c31bc483f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:13.534696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.544349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 446/911] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:13.567456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9c343-61b9-43f5-9526-823baa7f7377 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.582929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.676873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9c343-61b9-43f5-9526-823baa7f7377 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.711472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce7a5d-1c2d-435d-bc5d-93881116b88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.712612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce7a5d-1c2d-435d-bc5d-93881116b88b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.713614 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce7a5d-1c2d-435d-bc5d-93881116b88b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.714065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce7a5d-1c2d-435d-bc5d-93881116b88b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.714932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce7a5d-1c2d-435d-bc5d-93881116b88b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.716485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce7a5d-1c2d-435d-bc5d-93881116b88b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-7bce7a5d-1c2d-435d-bc5d-93881116b88b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:13.728963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:13.729948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:13.730835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:13.733368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 466/912] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:13.739460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945a7ad-f317-4ec5-85d3-44c31bc483f5 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.844236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945a7ad-f317-4ec5-85d3-44c31bc483f5 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.879491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0000d-f0dc-4d2b-87c5-d22947819c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.880441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0000d-f0dc-4d2b-87c5-d22947819c66 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.880864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0000d-f0dc-4d2b-87c5-d22947819c66 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.881324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0000d-f0dc-4d2b-87c5-d22947819c66 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.882212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0000d-f0dc-4d2b-87c5-d22947819c66 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:13.882688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0000d-f0dc-4d2b-87c5-d22947819c66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.884188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0000d-f0dc-4d2b-87c5-d22947819c66 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.894295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 447/913] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:13.896540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce7a5d-1c2d-435d-bc5d-93881116b88b None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.990866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.003385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce7a5d-1c2d-435d-bc5d-93881116b88b None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.037990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d7c03-6c97-4f48-8b3f-04b1da906371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.038944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d7c03-6c97-4f48-8b3f-04b1da906371 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.039375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d7c03-6c97-4f48-8b3f-04b1da906371 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.039791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d7c03-6c97-4f48-8b3f-04b1da906371 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.040662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d7c03-6c97-4f48-8b3f-04b1da906371 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:14.041180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d7c03-6c97-4f48-8b3f-04b1da906371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.042711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d7c03-6c97-4f48-8b3f-04b1da906371 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.051975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 467/914] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:14.053878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0000d-f0dc-4d2b-87c5-d22947819c66 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.114987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.157843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0000d-f0dc-4d2b-87c5-d22947819c66 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.193223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c6126-cb0a-4dac-87a3-119412d30aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.194135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c6126-cb0a-4dac-87a3-119412d30aff None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.194587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c6126-cb0a-4dac-87a3-119412d30aff None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.194999 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c6126-cb0a-4dac-87a3-119412d30aff None admin] PATH_INFO: `/v3/endpoints` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.195912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c6126-cb0a-4dac-87a3-119412d30aff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.197491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c6126-cb0a-4dac-87a3-119412d30aff None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.222014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b2b4-8fec-46e5-af5e-05a05a7377ec None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.222307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b2b4-8fec-46e5-af5e-05a05a7377ec None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.222624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b2b4-8fec-46e5-af5e-05a05a7377ec None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.224049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 468/915] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:14.229715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84f7aa-1b40-4bb6-b697-6e6fccc72c64 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.230007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84f7aa-1b40-4bb6-b697-6e6fccc72c64 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.230391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84f7aa-1b40-4bb6-b697-6e6fccc72c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.248015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-7b84f7aa-1b40-4bb6-b697-6e6fccc72c64 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:14.249209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7b84f7aa-1b40-4bb6-b697-6e6fccc72c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:14.279076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 448/916] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 05:41:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:14.342592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 469/917] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:14.351261 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a10fca-05f6-46d4-a6d3-52f389e66cd4 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.351582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a10fca-05f6-46d4-a6d3-52f389e66cd4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.351824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a10fca-05f6-46d4-a6d3-52f389e66cd4 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.355046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 449/918] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:14.359903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6546b3-03ae-433e-8427-bc7789e1397c None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.360198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6546b3-03ae-433e-8427-bc7789e1397c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.360480 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6546b3-03ae-433e-8427-bc7789e1397c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.366638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a10fca-05f6-46d4-a6d3-52f389e66cd4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.382794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-7e6546b3-03ae-433e-8427-bc7789e1397c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:14.384016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7e6546b3-03ae-433e-8427-bc7789e1397c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:14.452274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.468626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 470/919] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:14.484492 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6546b3-03ae-433e-8427-bc7789e1397c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.496770 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a10fca-05f6-46d4-a6d3-52f389e66cd4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.512801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.543308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d6714-2ebb-4f5c-8df3-f26358c3a0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.543978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d6714-2ebb-4f5c-8df3-f26358c3a0c6 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.544256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d6714-2ebb-4f5c-8df3-f26358c3a0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.544528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d6714-2ebb-4f5c-8df3-f26358c3a0c6 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.545255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d6714-2ebb-4f5c-8df3-f26358c3a0c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.546680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d6714-2ebb-4f5c-8df3-f26358c3a0c6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.602637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6546b3-03ae-433e-8427-bc7789e1397c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.640563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4f450-1904-4813-85a7-d44c93adf8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.641284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4f450-1904-4813-85a7-d44c93adf8e0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.641565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4f450-1904-4813-85a7-d44c93adf8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.641843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4f450-1904-4813-85a7-d44c93adf8e0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.642530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4f450-1904-4813-85a7-d44c93adf8e0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.643915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4f450-1904-4813-85a7-d44c93adf8e0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-b6d4f450-1904-4813-85a7-d44c93adf8e0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:14.655804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:14.656838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:14.657686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 471/920] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:14.663075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4f450-1904-4813-85a7-d44c93adf8e0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.666080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 450/921] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:41:14 2022] POST /identity/v3/services => generated 241 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:14.750585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.798343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4f450-1904-4813-85a7-d44c93adf8e0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.841786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910dab0-40c0-4596-abaf-bca7d338ae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.842829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910dab0-40c0-4596-abaf-bca7d338ae51 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.843317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910dab0-40c0-4596-abaf-bca7d338ae51 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.843825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910dab0-40c0-4596-abaf-bca7d338ae51 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.844734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910dab0-40c0-4596-abaf-bca7d338ae51 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:14.845243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910dab0-40c0-4596-abaf-bca7d338ae51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.846917 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910dab0-40c0-4596-abaf-bca7d338ae51 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.852160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 472/922] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:14.858667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d6714-2ebb-4f5c-8df3-f26358c3a0c6 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.967652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d6714-2ebb-4f5c-8df3-f26358c3a0c6 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:15.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eee93-293c-4937-9312-f84679571658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:15.007190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eee93-293c-4937-9312-f84679571658 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:15.007550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eee93-293c-4937-9312-f84679571658 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:15.007825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eee93-293c-4937-9312-f84679571658 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:15.008538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eee93-293c-4937-9312-f84679571658 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:15.008919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eee93-293c-4937-9312-f84679571658 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:15.010880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eee93-293c-4937-9312-f84679571658 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:15.017132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 451/923] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:15.022733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910dab0-40c0-4596-abaf-bca7d338ae51 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:15.129860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910dab0-40c0-4596-abaf-bca7d338ae51 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:15.148398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:15.166281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48145f3-90a9-4d32-9d8b-9c680571ba45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:15.167235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48145f3-90a9-4d32-9d8b-9c680571ba45 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:15.167712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48145f3-90a9-4d32-9d8b-9c680571ba45 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:15.168215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48145f3-90a9-4d32-9d8b-9c680571ba45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:15.169159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48145f3-90a9-4d32-9d8b-9c680571ba45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:15.170815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48145f3-90a9-4d32-9d8b-9c680571ba45 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:15.283675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 473/924] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 05:41:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:41:15.587561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828de2d3-f35a-46a5-aab0-82c73e9ce337 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:15.588129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828de2d3-f35a-46a5-aab0-82c73e9ce337 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:15.588129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828de2d3-f35a-46a5-aab0-82c73e9ce337 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:15.589736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 452/925] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:15.594910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3efb39-a7ac-4f8d-8f75-b7b2be73e27e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:15.595321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3efb39-a7ac-4f8d-8f75-b7b2be73e27e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:15.595493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3efb39-a7ac-4f8d-8f75-b7b2be73e27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:15.612589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-7e3efb39-a7ac-4f8d-8f75-b7b2be73e27e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:15.613782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7e3efb39-a7ac-4f8d-8f75-b7b2be73e27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:15.691348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 474/926] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:15.713423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828de2d3-f35a-46a5-aab0-82c73e9ce337 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:15.757545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:15.821415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828de2d3-f35a-46a5-aab0-82c73e9ce337 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:15.861660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa313789-6804-46d7-b44b-f3930b88cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:15.862494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa313789-6804-46d7-b44b-f3930b88cabe None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:15.862791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa313789-6804-46d7-b44b-f3930b88cabe None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:15.863065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa313789-6804-46d7-b44b-f3930b88cabe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:15.874133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa313789-6804-46d7-b44b-f3930b88cabe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-aa313789-6804-46d7-b44b-f3930b88cabe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:15.882254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:15.883331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:15.884259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 453/927] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 05:41:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:15.897273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3efb39-a7ac-4f8d-8f75-b7b2be73e27e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:16.006392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3efb39-a7ac-4f8d-8f75-b7b2be73e27e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:16.015158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:16.043404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a9a3c-e98b-4b4c-a308-25c6e1eecccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:16.044087 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a9a3c-e98b-4b4c-a308-25c6e1eecccb None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.044371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a9a3c-e98b-4b4c-a308-25c6e1eecccb None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.044650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a9a3c-e98b-4b4c-a308-25c6e1eecccb None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.045408 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a9a3c-e98b-4b4c-a308-25c6e1eecccb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:16.045714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a9a3c-e98b-4b4c-a308-25c6e1eecccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:16.047179 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a9a3c-e98b-4b4c-a308-25c6e1eecccb None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:16.056035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 475/928] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 05:41:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:16.061868 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa313789-6804-46d7-b44b-f3930b88cabe None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:16.167650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:16.171363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa313789-6804-46d7-b44b-f3930b88cabe None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:16.204690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3c4a1-0e5b-415b-b69b-3ca3e095b8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:16.205488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3c4a1-0e5b-415b-b69b-3ca3e095b8b1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.205766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3c4a1-0e5b-415b-b69b-3ca3e095b8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.206035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3c4a1-0e5b-415b-b69b-3ca3e095b8b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.217222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3c4a1-0e5b-415b-b69b-3ca3e095b8b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:16.218757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3c4a1-0e5b-415b-b69b-3ca3e095b8b1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-62e3c4a1-0e5b-415b-b69b-3ca3e095b8b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:16.236353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:16.237285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:16.238201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:16.238201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:16.238201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:16.238201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:16.238201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 454/929] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:16.243191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a9a3c-e98b-4b4c-a308-25c6e1eecccb None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:16.349310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a9a3c-e98b-4b4c-a308-25c6e1eecccb None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:16.385856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbfc26-7ffc-4304-8dfb-2a225c34c00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:16.386548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbfc26-7ffc-4304-8dfb-2a225c34c00b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.386836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbfc26-7ffc-4304-8dfb-2a225c34c00b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.387115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbfc26-7ffc-4304-8dfb-2a225c34c00b None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.387840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbfc26-7ffc-4304-8dfb-2a225c34c00b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:16.388146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbfc26-7ffc-4304-8dfb-2a225c34c00b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:16.389785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbfc26-7ffc-4304-8dfb-2a225c34c00b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:16.403239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 476/930] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:16.408740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3c4a1-0e5b-415b-b69b-3ca3e095b8b1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:16.511027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b287614-4a7c-4e59-bd4b-a961f3c1d968 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.511327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b287614-4a7c-4e59-bd4b-a961f3c1d968 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.511499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b287614-4a7c-4e59-bd4b-a961f3c1d968 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.514381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 477/931] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:16.518322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4287-bb53-41da-a96d-f340db236779 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.518768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4287-bb53-41da-a96d-f340db236779 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.519051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c4287-bb53-41da-a96d-f340db236779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.519264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3c4a1-0e5b-415b-b69b-3ca3e095b8b1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:16.536309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-332c4287-bb53-41da-a96d-f340db236779 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:16.537883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-332c4287-bb53-41da-a96d-f340db236779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:16.555786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58615f4-1442-4ee7-9490-87e90826d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:16.556882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58615f4-1442-4ee7-9490-87e90826d635 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.557231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58615f4-1442-4ee7-9490-87e90826d635 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.557591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58615f4-1442-4ee7-9490-87e90826d635 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.563788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:16.566108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58615f4-1442-4ee7-9490-87e90826d635 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:16.567702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58615f4-1442-4ee7-9490-87e90826d635 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-a58615f4-1442-4ee7-9490-87e90826d635 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:16.583530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:16.585547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:16.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 455/932] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 05:41:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:16.596157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58615f4-1442-4ee7-9490-87e90826d635 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:16.615348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:16.624721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 478/933] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:16.648303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c4287-bb53-41da-a96d-f340db236779 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:16.726734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58615f4-1442-4ee7-9490-87e90826d635 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:16.768826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c4287-bb53-41da-a96d-f340db236779 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:16.772794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d176d91-0fba-4020-8730-ad7ee62326d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:16.773688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d176d91-0fba-4020-8730-ad7ee62326d2 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.774180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d176d91-0fba-4020-8730-ad7ee62326d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.774689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d176d91-0fba-4020-8730-ad7ee62326d2 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.776303 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d176d91-0fba-4020-8730-ad7ee62326d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:16.776800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d176d91-0fba-4020-8730-ad7ee62326d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:16.778887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d176d91-0fba-4020-8730-ad7ee62326d2 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:16.806325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 456/934] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 05:41:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:16.806325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93586c-82e9-4688-bcf7-4aeb46746099 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.806325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93586c-82e9-4688-bcf7-4aeb46746099 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.807212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93586c-82e9-4688-bcf7-4aeb46746099 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.808509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 457/935] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:16.809091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93586c-82e9-4688-bcf7-4aeb46746099 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:16.814929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790efe9-767a-45e1-8356-a078d5be9ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:16.815741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790efe9-767a-45e1-8356-a078d5be9ee5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.816029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790efe9-767a-45e1-8356-a078d5be9ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.816307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790efe9-767a-45e1-8356-a078d5be9ee5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.816981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790efe9-767a-45e1-8356-a078d5be9ee5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:16.818375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790efe9-767a-45e1-8356-a078d5be9ee5 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-3790efe9-767a-45e1-8356-a078d5be9ee5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:16.831101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:16.832159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:16.833013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 479/936] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:16.833828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704449b-3cc4-4386-96b1-2fdd93fce624 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.833828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704449b-3cc4-4386-96b1-2fdd93fce624 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.834922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704449b-3cc4-4386-96b1-2fdd93fce624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.850142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-c704449b-3cc4-4386-96b1-2fdd93fce624 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:16.851360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c704449b-3cc4-4386-96b1-2fdd93fce624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:16.861950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:16.893681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:16.928305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 480/937] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:16.929365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93586c-82e9-4688-bcf7-4aeb46746099 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:16.935422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704449b-3cc4-4386-96b1-2fdd93fce624 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:16.972650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd35fc-d774-47e4-b961-faf1ae8ce937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:16.973451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd35fc-d774-47e4-b961-faf1ae8ce937 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:16.973736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd35fc-d774-47e4-b961-faf1ae8ce937 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:16.974015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd35fc-d774-47e4-b961-faf1ae8ce937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:16.984578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd35fc-d774-47e4-b961-faf1ae8ce937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:16.986134 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd35fc-d774-47e4-b961-faf1ae8ce937 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-fcdd35fc-d774-47e4-b961-faf1ae8ce937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:17.005500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:17.006536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:17.007475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:17.007475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:17.007475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:17.007475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:17.007475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 458/938] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:17.007769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd35fc-d774-47e4-b961-faf1ae8ce937 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.064338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704449b-3cc4-4386-96b1-2fdd93fce624 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.100785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd374e-f945-49f0-a524-d9a7e42513eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.101460 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd374e-f945-49f0-a524-d9a7e42513eb None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.101746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd374e-f945-49f0-a524-d9a7e42513eb None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.102024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd374e-f945-49f0-a524-d9a7e42513eb None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.102772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd374e-f945-49f0-a524-d9a7e42513eb None admin] RBAC: Adding query filter params (name=compute) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:17.103080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd374e-f945-49f0-a524-d9a7e42513eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.104490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd374e-f945-49f0-a524-d9a7e42513eb None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.108996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd35fc-d774-47e4-b961-faf1ae8ce937 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.114072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 481/939] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:17.114072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd374e-f945-49f0-a524-d9a7e42513eb None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.127350 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:17.135743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:17.151079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b789e-14c8-4f1e-8b52-3ac207697a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.151396 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b789e-14c8-4f1e-8b52-3ac207697a8d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.151687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b789e-14c8-4f1e-8b52-3ac207697a8d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.151968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b789e-14c8-4f1e-8b52-3ac207697a8d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.152622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b789e-14c8-4f1e-8b52-3ac207697a8d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.153995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b789e-14c8-4f1e-8b52-3ac207697a8d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-3e2b789e-14c8-4f1e-8b52-3ac207697a8d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:17.166898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:17.167944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:17.168814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 459/940] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:17.170138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b789e-14c8-4f1e-8b52-3ac207697a8d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.223826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd374e-f945-49f0-a524-d9a7e42513eb None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.259796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e26e1a-2702-40e5-98f5-455cb92404ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.260467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e26e1a-2702-40e5-98f5-455cb92404ab None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.260752 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e26e1a-2702-40e5-98f5-455cb92404ab None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.261032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e26e1a-2702-40e5-98f5-455cb92404ab None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.261768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e26e1a-2702-40e5-98f5-455cb92404ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:17.262077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e26e1a-2702-40e5-98f5-455cb92404ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.263490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e26e1a-2702-40e5-98f5-455cb92404ab None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.276120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 482/941] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:17.276938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e26e1a-2702-40e5-98f5-455cb92404ab None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.295696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b789e-14c8-4f1e-8b52-3ac207697a8d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.330386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557a6f2-b61f-4f04-9928-ebd34a279358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.331007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557a6f2-b61f-4f04-9928-ebd34a279358 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.331288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557a6f2-b61f-4f04-9928-ebd34a279358 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.331565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557a6f2-b61f-4f04-9928-ebd34a279358 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.332266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557a6f2-b61f-4f04-9928-ebd34a279358 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:17.332574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557a6f2-b61f-4f04-9928-ebd34a279358 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.333950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557a6f2-b61f-4f04-9928-ebd34a279358 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.338452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 460/942] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:17.339258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557a6f2-b61f-4f04-9928-ebd34a279358 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.370095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:17.406996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e26e1a-2702-40e5-98f5-455cb92404ab None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.415637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:17.445404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebdc88-c433-4ff9-8015-20417b926149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.446209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebdc88-c433-4ff9-8015-20417b926149 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.446519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebdc88-c433-4ff9-8015-20417b926149 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.446802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebdc88-c433-4ff9-8015-20417b926149 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.448255 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebdc88-c433-4ff9-8015-20417b926149 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:17.448728 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebdc88-c433-4ff9-8015-20417b926149 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.450463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebdc88-c433-4ff9-8015-20417b926149 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.461572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 483/943] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:17.461572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae00c21-a911-4522-b69c-6e45be0e0c51 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.461572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae00c21-a911-4522-b69c-6e45be0e0c51 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.461572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae00c21-a911-4522-b69c-6e45be0e0c51 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.461572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 484/944] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:17.465941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae00c21-a911-4522-b69c-6e45be0e0c51 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.476830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557a6f2-b61f-4f04-9928-ebd34a279358 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.519061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5d968-dc03-40ca-b4b8-3596c7cc1b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.519874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5d968-dc03-40ca-b4b8-3596c7cc1b14 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.520160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5d968-dc03-40ca-b4b8-3596c7cc1b14 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.520439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5d968-dc03-40ca-b4b8-3596c7cc1b14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.528159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5d968-dc03-40ca-b4b8-3596c7cc1b14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.535920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5d968-dc03-40ca-b4b8-3596c7cc1b14 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-27c5d968-dc03-40ca-b4b8-3596c7cc1b14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:17.544367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:17.545317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:17.546252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:17.546252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:17.546252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:17.546252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:17.546252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:17.546252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:17.546252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:17.546252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:17.546252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 461/945] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:17.546812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370d189-5cef-45de-9651-3efeb57bc291 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.547254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370d189-5cef-45de-9651-3efeb57bc291 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.547656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370d189-5cef-45de-9651-3efeb57bc291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.565399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-6370d189-5cef-45de-9651-3efeb57bc291 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:17.567145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6370d189-5cef-45de-9651-3efeb57bc291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:17.601129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae00c21-a911-4522-b69c-6e45be0e0c51 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.639620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59208c06-92cf-49f2-9331-48a6862540ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.640307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208c06-92cf-49f2-9331-48a6862540ae None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.640552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208c06-92cf-49f2-9331-48a6862540ae None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.640822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208c06-92cf-49f2-9331-48a6862540ae None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.641525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59208c06-92cf-49f2-9331-48a6862540ae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:17.641827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59208c06-92cf-49f2-9331-48a6862540ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.643234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59208c06-92cf-49f2-9331-48a6862540ae None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.644933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:17.653846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 485/946] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:17.653846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59208c06-92cf-49f2-9331-48a6862540ae None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.654503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 462/947] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:17.655274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370d189-5cef-45de-9651-3efeb57bc291 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.698996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:17.757348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370d189-5cef-45de-9651-3efeb57bc291 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.766700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59208c06-92cf-49f2-9331-48a6862540ae None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.793052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b36e4-028f-4a5d-bd93-874ac88b95d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.793707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b36e4-028f-4a5d-bd93-874ac88b95d4 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.793985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b36e4-028f-4a5d-bd93-874ac88b95d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.794263 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b36e4-028f-4a5d-bd93-874ac88b95d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.795072 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b36e4-028f-4a5d-bd93-874ac88b95d4 None admin] RBAC: Adding query filter params (interface=public, service_id=e5d8db42db5845abbf4b7c148c2d2eea, region_id=RegionOne) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:17.795379 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b36e4-028f-4a5d-bd93-874ac88b95d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.796738 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b36e4-028f-4a5d-bd93-874ac88b95d4 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.806962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 463/948] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/endpoints?service_id=e5d8db42db5845abbf4b7c148c2d2eea&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:17.806962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b36e4-028f-4a5d-bd93-874ac88b95d4 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.808664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448049f-b08c-4fa9-92b3-ba16191dcda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.809343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448049f-b08c-4fa9-92b3-ba16191dcda2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.809628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448049f-b08c-4fa9-92b3-ba16191dcda2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.809906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448049f-b08c-4fa9-92b3-ba16191dcda2 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.810747 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448049f-b08c-4fa9-92b3-ba16191dcda2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:17.811062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448049f-b08c-4fa9-92b3-ba16191dcda2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.812999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448049f-b08c-4fa9-92b3-ba16191dcda2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.831387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 486/949] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:17.831387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448049f-b08c-4fa9-92b3-ba16191dcda2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.901552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:17.928126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b36e4-028f-4a5d-bd93-874ac88b95d4 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.929663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:17.951894 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448049f-b08c-4fa9-92b3-ba16191dcda2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.966624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.967821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.968552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.968987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.970356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.972291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.973681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WARNING keystone.server.flask.common [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 05:41:17.989806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff16fd-df31-44df-b651-9eec47fe1143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.990618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff16fd-df31-44df-b651-9eec47fe1143 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.990905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff16fd-df31-44df-b651-9eec47fe1143 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.991207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff16fd-df31-44df-b651-9eec47fe1143 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.991997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff16fd-df31-44df-b651-9eec47fe1143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.993384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff16fd-df31-44df-b651-9eec47fe1143 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.000437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 487/950] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:41:18.001124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff16fd-df31-44df-b651-9eec47fe1143 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.039305 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.sql.core [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 05:41:18.039305 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 05:41:18.039305 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [parameters: {'id': '2963e22167d6447db270279928c2f31d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 05:41:18.039305 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91980) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2963e22167d6447db270279928c2f31d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2963e22167d6447db270279928c2f31d. Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:41:18.057014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application result.read() Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 25 05:41:18.057907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application yield resource Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise self.value Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application session.commit() Jul 25 05:41:18.058933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self.session.flush() Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application compat.raise_( Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 05:41:18.059854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 05:41:18.060818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application result.read() Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:41:18.061653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [parameters: {'id': '2963e22167d6447db270279928c2f31d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:18.064003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:18.066687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2963e22167d6447db270279928c2f31d. Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 464/951] 10.222.0.17 () {60 vars in 1140 bytes} [Mon Jul 25 05:41:17 2022] POST /identity/v3/projects => generated 262 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:18.068493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.154847 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff16fd-df31-44df-b651-9eec47fe1143 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.182210 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:18.185350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad7c78-8769-4d2d-ac67-fe71a10db79a None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.199010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83475f35-88a3-49ba-a604-b2dcf1c41467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.199616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83475f35-88a3-49ba-a604-b2dcf1c41467 None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.199885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83475f35-88a3-49ba-a604-b2dcf1c41467 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.200287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83475f35-88a3-49ba-a604-b2dcf1c41467 None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/56d11b3ed0474760a10e6f94df838297/roles/333abb19b7ef46c7a17f4f86c8d798e3` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.202732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83475f35-88a3-49ba-a604-b2dcf1c41467 None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=56d11b3ed0474760a10e6f94df838297, role_id=333abb19b7ef46c7a17f4f86c8d798e3)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.204186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83475f35-88a3-49ba-a604-b2dcf1c41467 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.230495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15c64b-5dab-4baf-acc9-eeb90da00c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.230495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15c64b-5dab-4baf-acc9-eeb90da00c27 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.230495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15c64b-5dab-4baf-acc9-eeb90da00c27 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.230495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15c64b-5dab-4baf-acc9-eeb90da00c27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.234426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15c64b-5dab-4baf-acc9-eeb90da00c27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.236193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15c64b-5dab-4baf-acc9-eeb90da00c27 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-8a15c64b-5dab-4baf-acc9-eeb90da00c27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:18.250998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:18.252001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:18.253037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:18.253037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:18.253037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:18.253037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:18.253037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:18.253037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:18.253037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:18.253037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:18.253037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 465/952] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:18.258626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15c64b-5dab-4baf-acc9-eeb90da00c27 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.266982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 488/953] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:18 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/56d11b3ed0474760a10e6f94df838297/roles/333abb19b7ef46c7a17f4f86c8d798e3 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:18.285721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:18.381523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15c64b-5dab-4baf-acc9-eeb90da00c27 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.420552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d061b-814c-45bc-950c-61dd96b41e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.421149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d061b-814c-45bc-950c-61dd96b41e61 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.421433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d061b-814c-45bc-950c-61dd96b41e61 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.421687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d061b-814c-45bc-950c-61dd96b41e61 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.422547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d061b-814c-45bc-950c-61dd96b41e61 None admin] RBAC: Adding query filter params (name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:18.422857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d061b-814c-45bc-950c-61dd96b41e61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.424370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d061b-814c-45bc-950c-61dd96b41e61 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.446825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 466/954] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:19.406038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863fac0-508f-4317-a305-d726950d436a None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:19.406719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863fac0-508f-4317-a305-d726950d436a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:19.406719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863fac0-508f-4317-a305-d726950d436a None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:19.408267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 489/955] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:19.413436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49293aea-8273-4fe5-b91a-b46723c45fa7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:19.413902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49293aea-8273-4fe5-b91a-b46723c45fa7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:19.414059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49293aea-8273-4fe5-b91a-b46723c45fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:19.431301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-49293aea-8273-4fe5-b91a-b46723c45fa7 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:19.432802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-49293aea-8273-4fe5-b91a-b46723c45fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:19.505366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 467/956] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:19.529138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863fac0-508f-4317-a305-d726950d436a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:19.637949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863fac0-508f-4317-a305-d726950d436a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:19.673125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca9583-3319-4631-ae59-026b4b9d70cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:19.673717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca9583-3319-4631-ae59-026b4b9d70cb None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:19.673998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca9583-3319-4631-ae59-026b4b9d70cb None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:19.674277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca9583-3319-4631-ae59-026b4b9d70cb None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:19.675047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca9583-3319-4631-ae59-026b4b9d70cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:19.676487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca9583-3319-4631-ae59-026b4b9d70cb None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:19.750751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 490/957] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:41:19 2022] POST /identity/v3/services => generated 242 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:20.117842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9f0fa-a26f-43e6-b976-9ed02fa9ede3 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.117842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9f0fa-a26f-43e6-b976-9ed02fa9ede3 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.117842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9f0fa-a26f-43e6-b976-9ed02fa9ede3 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.119350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 468/958] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:20.126525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de206-dc65-4116-963a-4dd2e1498626 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.126525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de206-dc65-4116-963a-4dd2e1498626 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.126525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de206-dc65-4116-963a-4dd2e1498626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.143853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9d6de206-dc65-4116-963a-4dd2e1498626 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:20.145322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9d6de206-dc65-4116-963a-4dd2e1498626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:20.190507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:20.224412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 491/959] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:20.248098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9f0fa-a26f-43e6-b976-9ed02fa9ede3 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:20.261256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:20.360397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9f0fa-a26f-43e6-b976-9ed02fa9ede3 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:20.399133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1244e-2ff6-49f8-9779-1923bbd2c7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:20.399867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1244e-2ff6-49f8-9779-1923bbd2c7a7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.400705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1244e-2ff6-49f8-9779-1923bbd2c7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.400984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1244e-2ff6-49f8-9779-1923bbd2c7a7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.401640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1244e-2ff6-49f8-9779-1923bbd2c7a7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:20.403048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1244e-2ff6-49f8-9779-1923bbd2c7a7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-fbc1244e-2ff6-49f8-9779-1923bbd2c7a7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:20.415706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:20.417218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:20.418115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:20.422079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 469/960] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:20.423498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de206-dc65-4116-963a-4dd2e1498626 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:20.432106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d036f-5f5b-4680-a73e-fb81c7c08d7d None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.432396 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d036f-5f5b-4680-a73e-fb81c7c08d7d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.432682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d036f-5f5b-4680-a73e-fb81c7c08d7d None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.434304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 470/961] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:20.439286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c7340-454a-4d6c-91bf-e9436e6af33e None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.439568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c7340-454a-4d6c-91bf-e9436e6af33e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.439842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c7340-454a-4d6c-91bf-e9436e6af33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.457484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-cd3c7340-454a-4d6c-91bf-e9436e6af33e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:20.459014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-cd3c7340-454a-4d6c-91bf-e9436e6af33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:20.538627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 471/962] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:20.543155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f81b0e-d6e8-4437-bf1d-881e7bfcab03 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.543442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f81b0e-d6e8-4437-bf1d-881e7bfcab03 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.543720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f81b0e-d6e8-4437-bf1d-881e7bfcab03 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.545303 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 472/963] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:20.549932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f7922-d449-4e84-943e-ad32c6a3129a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.550222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f7922-d449-4e84-943e-ad32c6a3129a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.550516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f7922-d449-4e84-943e-ad32c6a3129a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.555687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:20.560566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de206-dc65-4116-963a-4dd2e1498626 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:20.569000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3c2f7922-d449-4e84-943e-ad32c6a3129a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:20.570182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3c2f7922-d449-4e84-943e-ad32c6a3129a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:20.596713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0252b24-a727-4b4a-a69d-75ab690c2a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:20.597372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0252b24-a727-4b4a-a69d-75ab690c2a9e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.597653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0252b24-a727-4b4a-a69d-75ab690c2a9e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.597924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0252b24-a727-4b4a-a69d-75ab690c2a9e None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.598658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0252b24-a727-4b4a-a69d-75ab690c2a9e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:20.598963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0252b24-a727-4b4a-a69d-75ab690c2a9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:20.600384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0252b24-a727-4b4a-a69d-75ab690c2a9e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:20.605842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 492/964] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 05:41:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:20.606725 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0252b24-a727-4b4a-a69d-75ab690c2a9e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:20.639428 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:20.647726 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 473/965] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:20.648636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f7922-d449-4e84-943e-ad32c6a3129a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:20.718180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0252b24-a727-4b4a-a69d-75ab690c2a9e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:20.764384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb530ae-a3a3-4e37-9e91-9e1238b43d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:20.765290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb530ae-a3a3-4e37-9e91-9e1238b43d1c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.765574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb530ae-a3a3-4e37-9e91-9e1238b43d1c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.765848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb530ae-a3a3-4e37-9e91-9e1238b43d1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.766454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f7922-d449-4e84-943e-ad32c6a3129a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:20.776701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb530ae-a3a3-4e37-9e91-9e1238b43d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-7eb530ae-a3a3-4e37-9e91-9e1238b43d1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:20.785294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:20.786279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 493/966] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 05:41:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:20.787505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb530ae-a3a3-4e37-9e91-9e1238b43d1c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:20.801518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:20.807386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59aa314-3547-46b8-a7b7-d67c1e632430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:20.808222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa314-3547-46b8-a7b7-d67c1e632430 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.808780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa314-3547-46b8-a7b7-d67c1e632430 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.809059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa314-3547-46b8-a7b7-d67c1e632430 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.809829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59aa314-3547-46b8-a7b7-d67c1e632430 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:20.810224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59aa314-3547-46b8-a7b7-d67c1e632430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:20.811949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59aa314-3547-46b8-a7b7-d67c1e632430 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:20.821348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 474/967] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 05:41:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:20.821348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59aa314-3547-46b8-a7b7-d67c1e632430 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:20.896812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:20.908951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb530ae-a3a3-4e37-9e91-9e1238b43d1c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:20.929282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59aa314-3547-46b8-a7b7-d67c1e632430 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:20.948485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:20.949159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.949551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.949819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.950539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:20.952082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:20.953498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: WARNING keystone.server.flask.common [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 05:41:20.963127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84792144-f126-4531-95cb-dd8693147987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:20.963936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84792144-f126-4531-95cb-dd8693147987 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.964105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84792144-f126-4531-95cb-dd8693147987 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.964379 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84792144-f126-4531-95cb-dd8693147987 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.965113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84792144-f126-4531-95cb-dd8693147987 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:20.965416 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84792144-f126-4531-95cb-dd8693147987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:20.966789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84792144-f126-4531-95cb-dd8693147987 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:20.975089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 475/968] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 05:41:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:20.975637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84792144-f126-4531-95cb-dd8693147987 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.002831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.sql.core [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 05:41:21.002831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 05:41:21.002831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [parameters: {'user_id': 'e7ab86ef99b04053b11dd591aed4bf31', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 05:41:21.002831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91981) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application result.read() Jul 25 05:41:21.010735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:41:21.011602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application yield resource Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise self.value Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application session.commit() Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 05:41:21.012608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self.session.flush() Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application compat.raise_( Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 05:41:21.014402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:41:21.015435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application result.read() Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:41:21.016418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e7ab86ef99b04053b11dd591aed4bf31', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:21.018044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 25 05:41:21.019097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 25 05:41:21.020239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 25 05:41:21.020239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 25 05:41:21.020239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 05:41:21.020239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 05:41:21.020239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 05:41:21.020239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:21.020239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 494/969] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Jul 25 05:41:20 2022] POST /identity/v3/users => generated 160 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:21.020239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.048263 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:21.095724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84792144-f126-4531-95cb-dd8693147987 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.122792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325ff86-8a2c-4fd4-b42f-ccd82a070970 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.131093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364d203-2d01-41e5-b76e-e27910471fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.131759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364d203-2d01-41e5-b76e-e27910471fdb None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.132245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364d203-2d01-41e5-b76e-e27910471fdb None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.132372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364d203-2d01-41e5-b76e-e27910471fdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.133129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364d203-2d01-41e5-b76e-e27910471fdb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:21.134583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364d203-2d01-41e5-b76e-e27910471fdb None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:21.157412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5304d-c2ab-45c2-b3dc-706e5c009486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.158208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5304d-c2ab-45c2-b3dc-706e5c009486 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.158518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5304d-c2ab-45c2-b3dc-706e5c009486 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.158792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5304d-c2ab-45c2-b3dc-706e5c009486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.169369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5304d-c2ab-45c2-b3dc-706e5c009486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:21.171062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5304d-c2ab-45c2-b3dc-706e5c009486 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-1cb5304d-c2ab-45c2-b3dc-706e5c009486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:21.189326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:21.190231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:21.191184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:21.191184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:21.191184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:21.191184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:21.191184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 495/970] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:21.191502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5304d-c2ab-45c2-b3dc-706e5c009486 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.200027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:21.204354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 476/971] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 05:41:20 2022] POST /identity/v3/endpoints => generated 349 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:21.205176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364d203-2d01-41e5-b76e-e27910471fdb None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.357507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364d203-2d01-41e5-b76e-e27910471fdb None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.370201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:21.371433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5304d-c2ab-45c2-b3dc-706e5c009486 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.397284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb982cb2-9126-4b69-b26a-01a09ce0deda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.397954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982cb2-9126-4b69-b26a-01a09ce0deda None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.398233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982cb2-9126-4b69-b26a-01a09ce0deda None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.398542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982cb2-9126-4b69-b26a-01a09ce0deda None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.399471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb982cb2-9126-4b69-b26a-01a09ce0deda None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:21.399779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb982cb2-9126-4b69-b26a-01a09ce0deda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:21.401680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb982cb2-9126-4b69-b26a-01a09ce0deda None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:21.416971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 477/972] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:21.422385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f775e4d-576f-43e7-ab86-65a70d91f13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.423170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb982cb2-9126-4b69-b26a-01a09ce0deda None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.423397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f775e4d-576f-43e7-ab86-65a70d91f13c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.423750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f775e4d-576f-43e7-ab86-65a70d91f13c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.424236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f775e4d-576f-43e7-ab86-65a70d91f13c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.433883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f775e4d-576f-43e7-ab86-65a70d91f13c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:21.435417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f775e4d-576f-43e7-ab86-65a70d91f13c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-8f775e4d-576f-43e7-ab86-65a70d91f13c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:21.462996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:21.464027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:21.464981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 496/973] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:21.473010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f775e4d-576f-43e7-ab86-65a70d91f13c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.548129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:21.556550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb982cb2-9126-4b69-b26a-01a09ce0deda None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.588458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f775e4d-576f-43e7-ab86-65a70d91f13c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.604616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee386d2-8372-4bcc-b8b0-81ba3c414cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.605974 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee386d2-8372-4bcc-b8b0-81ba3c414cb1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.606259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee386d2-8372-4bcc-b8b0-81ba3c414cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.606569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee386d2-8372-4bcc-b8b0-81ba3c414cb1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.623348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee386d2-8372-4bcc-b8b0-81ba3c414cb1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:21.626070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee386d2-8372-4bcc-b8b0-81ba3c414cb1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:21.628824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5121993-5003-45d0-ad77-b9f41b9c4e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.629759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5121993-5003-45d0-ad77-b9f41b9c4e9c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.630315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5121993-5003-45d0-ad77-b9f41b9c4e9c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.630816 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5121993-5003-45d0-ad77-b9f41b9c4e9c None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.631821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5121993-5003-45d0-ad77-b9f41b9c4e9c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:21.632327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5121993-5003-45d0-ad77-b9f41b9c4e9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:21.633970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5121993-5003-45d0-ad77-b9f41b9c4e9c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-0ee386d2-8372-4bcc-b8b0-81ba3c414cb1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:21.649417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:21.651068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:21.652068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:21.652068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:21.652068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:21.652068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:21.652068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:21.652068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:21.652068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:21.652068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:21.652068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:21.652068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:21.652636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 478/974] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:21.659444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee386d2-8372-4bcc-b8b0-81ba3c414cb1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.691560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:21.696014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 497/975] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:21.789144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee386d2-8372-4bcc-b8b0-81ba3c414cb1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.826594 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a62fb-2823-4456-8c76-a04c6a48db96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.827296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a62fb-2823-4456-8c76-a04c6a48db96 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.827579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a62fb-2823-4456-8c76-a04c6a48db96 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.828036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a62fb-2823-4456-8c76-a04c6a48db96 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.828904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a62fb-2823-4456-8c76-a04c6a48db96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:21.829216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a62fb-2823-4456-8c76-a04c6a48db96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:21.830703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a62fb-2823-4456-8c76-a04c6a48db96 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:21.852113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 479/976] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:21.857552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5121993-5003-45d0-ad77-b9f41b9c4e9c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.866921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486434e8-6b94-4d31-ac98-c3c1f0e2bac5 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.867240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486434e8-6b94-4d31-ac98-c3c1f0e2bac5 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.867473 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486434e8-6b94-4d31-ac98-c3c1f0e2bac5 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.869171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 480/977] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:21.874015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96ade-4bf8-47af-8210-6a480abe21a2 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.874235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96ade-4bf8-47af-8210-6a480abe21a2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.874547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96ade-4bf8-47af-8210-6a480abe21a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.892558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-d4b96ade-4bf8-47af-8210-6a480abe21a2 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:21.893752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d4b96ade-4bf8-47af-8210-6a480abe21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:21.979425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 481/978] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:21.987330 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5121993-5003-45d0-ad77-b9f41b9c4e9c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.001366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b96ade-4bf8-47af-8210-6a480abe21a2 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.016898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.024786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.029186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3652540-962d-4d49-9c4f-7060dc77df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.033501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3652540-962d-4d49-9c4f-7060dc77df7d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.033501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3652540-962d-4d49-9c4f-7060dc77df7d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.033501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3652540-962d-4d49-9c4f-7060dc77df7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.111795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3652540-962d-4d49-9c4f-7060dc77df7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.112303 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b96ade-4bf8-47af-8210-6a480abe21a2 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.113267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3652540-962d-4d49-9c4f-7060dc77df7d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.150621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e161-e49f-44b7-820e-55b1d2ef4ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.151348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e161-e49f-44b7-820e-55b1d2ef4ab1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.151624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e161-e49f-44b7-820e-55b1d2ef4ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.151892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e161-e49f-44b7-820e-55b1d2ef4ab1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.152534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127e161-e49f-44b7-820e-55b1d2ef4ab1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.154372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127e161-e49f-44b7-820e-55b1d2ef4ab1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-2127e161-e49f-44b7-820e-55b1d2ef4ab1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:22.171455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:22.172466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:41:22.173334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:22.174943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 482/979] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:22.180375 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e161-e49f-44b7-820e-55b1d2ef4ab1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.217288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 498/980] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:22.222903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3652540-962d-4d49-9c4f-7060dc77df7d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.248619 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.275321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.293151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e161-e49f-44b7-820e-55b1d2ef4ab1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.326814 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3652540-962d-4d49-9c4f-7060dc77df7d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.329120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3435f5-7b1e-4773-81aa-df760d5440f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.329866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3435f5-7b1e-4773-81aa-df760d5440f7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.330215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3435f5-7b1e-4773-81aa-df760d5440f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.330586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3435f5-7b1e-4773-81aa-df760d5440f7 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.331357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3435f5-7b1e-4773-81aa-df760d5440f7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:22.331732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3435f5-7b1e-4773-81aa-df760d5440f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.333150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3435f5-7b1e-4773-81aa-df760d5440f7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.342597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 483/981] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:22.343964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3435f5-7b1e-4773-81aa-df760d5440f7 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.372208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ab497-0588-46fe-bbb8-f61a1bf76248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.373182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ab497-0588-46fe-bbb8-f61a1bf76248 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.373561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ab497-0588-46fe-bbb8-f61a1bf76248 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.373941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ab497-0588-46fe-bbb8-f61a1bf76248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.387517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ab497-0588-46fe-bbb8-f61a1bf76248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.389534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ab497-0588-46fe-bbb8-f61a1bf76248 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-ad3ab497-0588-46fe-bbb8-f61a1bf76248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:22.408665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:22.409861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:22.411466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:22.411466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:22.411466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:22.411466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:22.411466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 499/982] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:22.416116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ab497-0588-46fe-bbb8-f61a1bf76248 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.456957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3435f5-7b1e-4773-81aa-df760d5440f7 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.494669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d022478-7054-461a-8b41-e2fff5da94ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.495442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022478-7054-461a-8b41-e2fff5da94ae None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.495720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022478-7054-461a-8b41-e2fff5da94ae None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.495988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022478-7054-461a-8b41-e2fff5da94ae None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.496685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d022478-7054-461a-8b41-e2fff5da94ae None admin] RBAC: Adding query filter params (type=compute) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:22.496988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d022478-7054-461a-8b41-e2fff5da94ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.498349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d022478-7054-461a-8b41-e2fff5da94ae None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.503078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.503536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 484/983] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:22.509268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d022478-7054-461a-8b41-e2fff5da94ae None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.522581 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.532230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ab497-0588-46fe-bbb8-f61a1bf76248 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.568409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5540f10-0e30-421d-b585-e862d16da979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.569067 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5540f10-0e30-421d-b585-e862d16da979 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.569352 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5540f10-0e30-421d-b585-e862d16da979 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.569634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5540f10-0e30-421d-b585-e862d16da979 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.570349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5540f10-0e30-421d-b585-e862d16da979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:22.570688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5540f10-0e30-421d-b585-e862d16da979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.572180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5540f10-0e30-421d-b585-e862d16da979 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.583603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 500/984] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:22.589074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5540f10-0e30-421d-b585-e862d16da979 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.611230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d022478-7054-461a-8b41-e2fff5da94ae None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.645082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29e60f-c2ce-43b1-a9e1-fdf843e19a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.645729 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29e60f-c2ce-43b1-a9e1-fdf843e19a1b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.646008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29e60f-c2ce-43b1-a9e1-fdf843e19a1b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.646300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29e60f-c2ce-43b1-a9e1-fdf843e19a1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.647125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29e60f-c2ce-43b1-a9e1-fdf843e19a1b None admin] RBAC: Adding query filter params (interface=public, service_id=c3a0691bee18424b89433a24285c989c, region_id=RegionOne) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:22.647418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29e60f-c2ce-43b1-a9e1-fdf843e19a1b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.648767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29e60f-c2ce-43b1-a9e1-fdf843e19a1b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.653935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 485/985] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/endpoints?service_id=c3a0691bee18424b89433a24285c989c&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:22.659283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29e60f-c2ce-43b1-a9e1-fdf843e19a1b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.702022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5540f10-0e30-421d-b585-e862d16da979 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.745431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b59b4-5075-46cc-9532-ea4a832a3654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.746229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b59b4-5075-46cc-9532-ea4a832a3654 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.746534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b59b4-5075-46cc-9532-ea4a832a3654 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.746815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b59b4-5075-46cc-9532-ea4a832a3654 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.749711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.750009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.754212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b59b4-5075-46cc-9532-ea4a832a3654 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.760641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b59b4-5075-46cc-9532-ea4a832a3654 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-ff6b59b4-5075-46cc-9532-ea4a832a3654 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:22.770923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:22.771965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:22.772871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:22.772871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:22.772871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:22.772871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:22.772871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:22.772871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:22.772871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:22.772871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:22.772871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 501/986] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:22.778729 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b59b4-5075-46cc-9532-ea4a832a3654 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.778897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29e60f-c2ce-43b1-a9e1-fdf843e19a1b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.817739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34214b76-6650-4768-9d56-3f5467c355b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.818412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34214b76-6650-4768-9d56-3f5467c355b9 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.818715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34214b76-6650-4768-9d56-3f5467c355b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.818988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34214b76-6650-4768-9d56-3f5467c355b9 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.819670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34214b76-6650-4768-9d56-3f5467c355b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.821021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34214b76-6650-4768-9d56-3f5467c355b9 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.827731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 486/987] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:41:22.886346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b59b4-5075-46cc-9532-ea4a832a3654 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.929163 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae6212-3117-4d01-9b40-d5d214ad2eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.929833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae6212-3117-4d01-9b40-d5d214ad2eaf None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.930115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae6212-3117-4d01-9b40-d5d214ad2eaf None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.930418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae6212-3117-4d01-9b40-d5d214ad2eaf None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.931299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae6212-3117-4d01-9b40-d5d214ad2eaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:22.931610 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae6212-3117-4d01-9b40-d5d214ad2eaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.933065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae6212-3117-4d01-9b40-d5d214ad2eaf None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.950545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 502/988] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:22.952662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34214b76-6650-4768-9d56-3f5467c355b9 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.069086 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34214b76-6650-4768-9d56-3f5467c355b9 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.103931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cdb08-567f-4b79-9bb5-d706e24711de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.104591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cdb08-567f-4b79-9bb5-d706e24711de None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.104877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cdb08-567f-4b79-9bb5-d706e24711de None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.105270 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cdb08-567f-4b79-9bb5-d706e24711de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.106094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cdb08-567f-4b79-9bb5-d706e24711de None admin] RBAC: Adding query filter params (role.id=333abb19b7ef46c7a17f4f86c8d798e3, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=56d11b3ed0474760a10e6f94df838297) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:23.106409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cdb08-567f-4b79-9bb5-d706e24711de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.107857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cdb08-567f-4b79-9bb5-d706e24711de None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.113949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 487/989] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/role_assignments?user.id=56d11b3ed0474760a10e6f94df838297&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=333abb19b7ef46c7a17f4f86c8d798e3 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:23.373893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02cf65-51d7-45a3-8b93-acc4182279f8 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.374294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02cf65-51d7-45a3-8b93-acc4182279f8 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.374493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02cf65-51d7-45a3-8b93-acc4182279f8 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.376017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 503/990] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:23.381022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4218c-b2ae-47db-abc2-a91ae166a78c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.381319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4218c-b2ae-47db-abc2-a91ae166a78c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.381615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4218c-b2ae-47db-abc2-a91ae166a78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.387300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.399883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-09d4218c-b2ae-47db-abc2-a91ae166a78c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:23.409231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-09d4218c-b2ae-47db-abc2-a91ae166a78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:23.468533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 488/991] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:23.490794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02cf65-51d7-45a3-8b93-acc4182279f8 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.548479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.595685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02cf65-51d7-45a3-8b93-acc4182279f8 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.629553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69782-1422-4920-92b8-628d4632dd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.630282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69782-1422-4920-92b8-628d4632dd7b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.630596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69782-1422-4920-92b8-628d4632dd7b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.630872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec69782-1422-4920-92b8-628d4632dd7b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.631524 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec69782-1422-4920-92b8-628d4632dd7b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.632923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec69782-1422-4920-92b8-628d4632dd7b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-dec69782-1422-4920-92b8-628d4632dd7b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:23.645661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:23.646662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:23.647539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 504/992] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:23.653090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4218c-b2ae-47db-abc2-a91ae166a78c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.751658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4218c-b2ae-47db-abc2-a91ae166a78c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.781105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.786512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cce56-aebc-435f-8538-da99a289b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.787208 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cce56-aebc-435f-8538-da99a289b606 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.787487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cce56-aebc-435f-8538-da99a289b606 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.787764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cce56-aebc-435f-8538-da99a289b606 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.788467 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410cce56-aebc-435f-8538-da99a289b606 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:23.788772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410cce56-aebc-435f-8538-da99a289b606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.790141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410cce56-aebc-435f-8538-da99a289b606 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.799626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 489/993] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:23.800829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69782-1422-4920-92b8-628d4632dd7b None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.908226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec69782-1422-4920-92b8-628d4632dd7b None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.933378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.947471 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e73547-d06a-40f4-b633-2d82b41cbce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.948160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73547-d06a-40f4-b633-2d82b41cbce1 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.948446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73547-d06a-40f4-b633-2d82b41cbce1 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.948727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73547-d06a-40f4-b633-2d82b41cbce1 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.949574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e73547-d06a-40f4-b633-2d82b41cbce1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:23.949887 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e73547-d06a-40f4-b633-2d82b41cbce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.951361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e73547-d06a-40f4-b633-2d82b41cbce1 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.957580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 505/994] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:23.963259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cce56-aebc-435f-8538-da99a289b606 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.065325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cce56-aebc-435f-8538-da99a289b606 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.100669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2f299-0883-4f20-85cc-e5bfb16077c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.101664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f299-0883-4f20-85cc-e5bfb16077c7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.102159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f299-0883-4f20-85cc-e5bfb16077c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.102664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f299-0883-4f20-85cc-e5bfb16077c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.104193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2f299-0883-4f20-85cc-e5bfb16077c7 None admin] RBAC: Adding query filter params (interface=admin, service_id=e5d8db42db5845abbf4b7c148c2d2eea, region_id=RegionOne) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:24.104709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2f299-0883-4f20-85cc-e5bfb16077c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.106288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2f299-0883-4f20-85cc-e5bfb16077c7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.117271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e73547-d06a-40f4-b633-2d82b41cbce1 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.117964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 490/995] 10.222.0.17 () {56 vars in 1249 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/endpoints?service_id=e5d8db42db5845abbf4b7c148c2d2eea&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:24.217771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e73547-d06a-40f4-b633-2d82b41cbce1 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.252859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a077a-df61-4d35-9b83-0394b110d647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.253526 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a077a-df61-4d35-9b83-0394b110d647 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.253807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a077a-df61-4d35-9b83-0394b110d647 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.254082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a077a-df61-4d35-9b83-0394b110d647 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.254800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a077a-df61-4d35-9b83-0394b110d647 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.256175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a077a-df61-4d35-9b83-0394b110d647 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.270577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 506/996] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 05:41:24 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:41:24.890094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c8ee6-038b-4ce7-a983-b3875d6c2aef None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.890640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c8ee6-038b-4ce7-a983-b3875d6c2aef None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.890750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c8ee6-038b-4ce7-a983-b3875d6c2aef None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.894338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 491/997] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:24.900920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a349db-e806-46e2-8041-7f65f90f5358 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.901366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a349db-e806-46e2-8041-7f65f90f5358 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.901515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a349db-e806-46e2-8041-7f65f90f5358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.922277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f1a349db-e806-46e2-8041-7f65f90f5358 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:24.923956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f1a349db-e806-46e2-8041-7f65f90f5358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:24.969901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:24.992800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 507/998] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:25.017441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c8ee6-038b-4ce7-a983-b3875d6c2aef None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.079842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.122548 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c8ee6-038b-4ce7-a983-b3875d6c2aef None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.154351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4b7d3-c29f-49a4-a5a6-9e6851c8ce2e None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.154661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4b7d3-c29f-49a4-a5a6-9e6851c8ce2e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.154866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4b7d3-c29f-49a4-a5a6-9e6851c8ce2e None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.156378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 508/999] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:25.157363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c046d-ff63-4618-bb92-090e0766d234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.158168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c046d-ff63-4618-bb92-090e0766d234 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.158457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c046d-ff63-4618-bb92-090e0766d234 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.158755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c046d-ff63-4618-bb92-090e0766d234 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.159406 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c046d-ff63-4618-bb92-090e0766d234 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.160863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c046d-ff63-4618-bb92-090e0766d234 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.160969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2f9c4-9e4d-4681-9e55-2a0fb9b6b874 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.161146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2f9c4-9e4d-4681-9e55-2a0fb9b6b874 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.161424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2f9c4-9e4d-4681-9e55-2a0fb9b6b874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-d10c046d-ff63-4618-bb92-090e0766d234 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:25.173077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:25.174119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:41:25.175097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:25.178782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 492/1000] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:25.180104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c046d-ff63-4618-bb92-090e0766d234 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.181911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-b5a2f9c4-9e4d-4681-9e55-2a0fb9b6b874 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:25.183138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b5a2f9c4-9e4d-4681-9e55-2a0fb9b6b874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:25.248862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 509/1001] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:25.271464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2f9c4-9e4d-4681-9e55-2a0fb9b6b874 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.283290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c046d-ff63-4618-bb92-090e0766d234 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.308923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.320362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43460c6c-5816-4444-8e90-4e8cef4e7da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.321094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460c6c-5816-4444-8e90-4e8cef4e7da6 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.321427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460c6c-5816-4444-8e90-4e8cef4e7da6 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.321798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43460c6c-5816-4444-8e90-4e8cef4e7da6 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.322994 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43460c6c-5816-4444-8e90-4e8cef4e7da6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:25.323306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43460c6c-5816-4444-8e90-4e8cef4e7da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.324671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43460c6c-5816-4444-8e90-4e8cef4e7da6 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.330565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 493/1002] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:25.335862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43460c6c-5816-4444-8e90-4e8cef4e7da6 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.380457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2f9c4-9e4d-4681-9e55-2a0fb9b6b874 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.391443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.423077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebca8e-9479-43c6-be43-c123ed5abd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.423791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebca8e-9479-43c6-be43-c123ed5abd36 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.424075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebca8e-9479-43c6-be43-c123ed5abd36 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.424354 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebca8e-9479-43c6-be43-c123ed5abd36 None admin] PATH_INFO: `/v3/services/image` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.425079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebca8e-9479-43c6-be43-c123ed5abd36 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.426588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebca8e-9479-43c6-be43-c123ed5abd36 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-97ebca8e-9479-43c6-be43-c123ed5abd36 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:25.439184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:25.440222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:25.441074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 510/1003] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:25.446280 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebca8e-9479-43c6-be43-c123ed5abd36 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.450596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43460c6c-5816-4444-8e90-4e8cef4e7da6 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.491649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3948fa1-b865-4ac3-b760-469c5a0d0bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.492723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3948fa1-b865-4ac3-b760-469c5a0d0bc5 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.493004 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3948fa1-b865-4ac3-b760-469c5a0d0bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.493569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3948fa1-b865-4ac3-b760-469c5a0d0bc5 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.495060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3948fa1-b865-4ac3-b760-469c5a0d0bc5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:25.495518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3948fa1-b865-4ac3-b760-469c5a0d0bc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.497175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3948fa1-b865-4ac3-b760-469c5a0d0bc5 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.504924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 494/1004] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:25.509535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3948fa1-b865-4ac3-b760-469c5a0d0bc5 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.555325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebca8e-9479-43c6-be43-c123ed5abd36 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.566513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.590342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428aa184-28b3-4e5a-80a5-60be13ff05fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.591044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aa184-28b3-4e5a-80a5-60be13ff05fa None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.591329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aa184-28b3-4e5a-80a5-60be13ff05fa None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.591609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428aa184-28b3-4e5a-80a5-60be13ff05fa None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.592324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428aa184-28b3-4e5a-80a5-60be13ff05fa None admin] RBAC: Adding query filter params (name=image) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:25.592631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428aa184-28b3-4e5a-80a5-60be13ff05fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.594071 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428aa184-28b3-4e5a-80a5-60be13ff05fa None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.604172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 511/1005] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:25.605773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428aa184-28b3-4e5a-80a5-60be13ff05fa None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.614662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3948fa1-b865-4ac3-b760-469c5a0d0bc5 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.643870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.652848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15074bee-7309-4d6a-bf2a-eba24c6715a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.653646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15074bee-7309-4d6a-bf2a-eba24c6715a8 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.653818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15074bee-7309-4d6a-bf2a-eba24c6715a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.654215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15074bee-7309-4d6a-bf2a-eba24c6715a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.654998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15074bee-7309-4d6a-bf2a-eba24c6715a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.656369 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15074bee-7309-4d6a-bf2a-eba24c6715a8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.710505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428aa184-28b3-4e5a-80a5-60be13ff05fa None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.714632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 495/1006] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 05:41:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:41:25.766550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a992054-1fd0-4e8a-9289-425c0a38ca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.767243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992054-1fd0-4e8a-9289-425c0a38ca20 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.767525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992054-1fd0-4e8a-9289-425c0a38ca20 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.767798 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a992054-1fd0-4e8a-9289-425c0a38ca20 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.768529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a992054-1fd0-4e8a-9289-425c0a38ca20 None admin] RBAC: Adding query filter params (type=image) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:25.768837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a992054-1fd0-4e8a-9289-425c0a38ca20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.770277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a992054-1fd0-4e8a-9289-425c0a38ca20 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.780244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 512/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:26.300909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e9dd0-9111-4fbe-a0d2-1f7aac74b71c None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.301447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e9dd0-9111-4fbe-a0d2-1f7aac74b71c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.301447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e9dd0-9111-4fbe-a0d2-1f7aac74b71c None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.302996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 496/1008] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:26.307959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16c4e3-8a2b-4e17-9d24-41242ca51310 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.308357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16c4e3-8a2b-4e17-9d24-41242ca51310 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.308521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16c4e3-8a2b-4e17-9d24-41242ca51310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.325932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-0c16c4e3-8a2b-4e17-9d24-41242ca51310 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:26.327134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0c16c4e3-8a2b-4e17-9d24-41242ca51310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:26.396634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 513/1009] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:26 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:26.418796 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e9dd0-9111-4fbe-a0d2-1f7aac74b71c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.521839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e9dd0-9111-4fbe-a0d2-1f7aac74b71c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.532871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.557429 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7959721-6d09-4b5e-acf5-7ceffed338ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.558164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7959721-6d09-4b5e-acf5-7ceffed338ee None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.558439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7959721-6d09-4b5e-acf5-7ceffed338ee None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.558730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7959721-6d09-4b5e-acf5-7ceffed338ee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.559370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7959721-6d09-4b5e-acf5-7ceffed338ee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.560742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7959721-6d09-4b5e-acf5-7ceffed338ee None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-f7959721-6d09-4b5e-acf5-7ceffed338ee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:26.573577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:26.574817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:41:26.575862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:26.582602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 497/1010] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:26.587961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16c4e3-8a2b-4e17-9d24-41242ca51310 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.613380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.684880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16c4e3-8a2b-4e17-9d24-41242ca51310 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.717930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06f66e-6864-49af-a754-68f26a953945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.718609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06f66e-6864-49af-a754-68f26a953945 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.718891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06f66e-6864-49af-a754-68f26a953945 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.719161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06f66e-6864-49af-a754-68f26a953945 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.719911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06f66e-6864-49af-a754-68f26a953945 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:26.720217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06f66e-6864-49af-a754-68f26a953945 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.721579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06f66e-6864-49af-a754-68f26a953945 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.730560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 514/1011] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:26.732446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7959721-6d09-4b5e-acf5-7ceffed338ee None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.836904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7959721-6d09-4b5e-acf5-7ceffed338ee None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.870098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ce280-595e-4387-8aa3-cc4b68b82b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.870931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ce280-595e-4387-8aa3-cc4b68b82b78 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.871351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ce280-595e-4387-8aa3-cc4b68b82b78 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.871723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ce280-595e-4387-8aa3-cc4b68b82b78 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.872519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ce280-595e-4387-8aa3-cc4b68b82b78 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:26.872936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ce280-595e-4387-8aa3-cc4b68b82b78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.874381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ce280-595e-4387-8aa3-cc4b68b82b78 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.884976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 498/1012] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:26.886665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06f66e-6864-49af-a754-68f26a953945 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.983205 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.992808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06f66e-6864-49af-a754-68f26a953945 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:27.029244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d37ec5-8e7f-435e-9c35-11fa2d55cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:27.029905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37ec5-8e7f-435e-9c35-11fa2d55cb67 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.030187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37ec5-8e7f-435e-9c35-11fa2d55cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.030479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d37ec5-8e7f-435e-9c35-11fa2d55cb67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.031228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d37ec5-8e7f-435e-9c35-11fa2d55cb67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:27.032630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d37ec5-8e7f-435e-9c35-11fa2d55cb67 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:27.128519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 515/1013] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 05:41:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:27.826246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01740455-ecb3-4679-9c97-90cc573508d1 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.826794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01740455-ecb3-4679-9c97-90cc573508d1 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.826872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01740455-ecb3-4679-9c97-90cc573508d1 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.828365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 499/1014] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:27.833322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bcd75-f3d5-4950-9b3d-d117d3461213 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.833696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bcd75-f3d5-4950-9b3d-d117d3461213 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.833880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bcd75-f3d5-4950-9b3d-d117d3461213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.851221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-450bcd75-f3d5-4950-9b3d-d117d3461213 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:27.852416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-450bcd75-f3d5-4950-9b3d-d117d3461213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:27.875951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870141-f3ae-4a84-b1ae-28e90dc754c7 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.876255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870141-f3ae-4a84-b1ae-28e90dc754c7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.876502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870141-f3ae-4a84-b1ae-28e90dc754c7 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.877999 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 500/1015] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:27.882741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc0ddb-5186-4c3b-aaa8-f94ed8f6ff51 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.883029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc0ddb-5186-4c3b-aaa8-f94ed8f6ff51 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.883307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc0ddb-5186-4c3b-aaa8-f94ed8f6ff51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.903663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-08fc0ddb-5186-4c3b-aaa8-f94ed8f6ff51 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:27.905662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-08fc0ddb-5186-4c3b-aaa8-f94ed8f6ff51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:27.929698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 516/1016] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:27 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:27.939337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:27.952597 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bcd75-f3d5-4950-9b3d-d117d3461213 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:27.980864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 501/1017] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:27 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:28.003627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc0ddb-5186-4c3b-aaa8-f94ed8f6ff51 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.039647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.063918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bcd75-f3d5-4950-9b3d-d117d3461213 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.100066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf099e-aadd-4839-873e-43885700229f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.100742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf099e-aadd-4839-873e-43885700229f None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.101026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf099e-aadd-4839-873e-43885700229f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.101304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf099e-aadd-4839-873e-43885700229f None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.102045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf099e-aadd-4839-873e-43885700229f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.103466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf099e-aadd-4839-873e-43885700229f None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.115365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc0ddb-5186-4c3b-aaa8-f94ed8f6ff51 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.148903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e73af-cbc7-48ab-b485-bdf883ebe416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.149703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e73af-cbc7-48ab-b485-bdf883ebe416 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.150147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e73af-cbc7-48ab-b485-bdf883ebe416 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.150274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e73af-cbc7-48ab-b485-bdf883ebe416 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.160271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e73af-cbc7-48ab-b485-bdf883ebe416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-011e73af-cbc7-48ab-b485-bdf883ebe416 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:28.168517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:28.169675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:28.170641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 502/1018] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:28.175638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e73af-cbc7-48ab-b485-bdf883ebe416 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.198923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 517/1019] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:41:27 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:41:28.208805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.292277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e73af-cbc7-48ab-b485-bdf883ebe416 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.335783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34829137-c36b-44fa-b8c0-5fe5201c5d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.336468 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34829137-c36b-44fa-b8c0-5fe5201c5d45 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.336833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34829137-c36b-44fa-b8c0-5fe5201c5d45 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.337027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34829137-c36b-44fa-b8c0-5fe5201c5d45 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.337816 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34829137-c36b-44fa-b8c0-5fe5201c5d45 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:28.338126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34829137-c36b-44fa-b8c0-5fe5201c5d45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.339546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34829137-c36b-44fa-b8c0-5fe5201c5d45 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.348893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 503/1020] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:28.354176 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf099e-aadd-4839-873e-43885700229f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.456019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf099e-aadd-4839-873e-43885700229f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.489274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680bb71-c774-4b7a-b39f-1cf8211a6ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.490063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680bb71-c774-4b7a-b39f-1cf8211a6ee0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.490340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680bb71-c774-4b7a-b39f-1cf8211a6ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.490718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680bb71-c774-4b7a-b39f-1cf8211a6ee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.493378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.502057 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680bb71-c774-4b7a-b39f-1cf8211a6ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.503539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680bb71-c774-4b7a-b39f-1cf8211a6ee0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-d680bb71-c774-4b7a-b39f-1cf8211a6ee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:28.521320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:28.522225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:28.523540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:28.523540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:28.523540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:28.523540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:28.523540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 518/1021] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:28.528393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34829137-c36b-44fa-b8c0-5fe5201c5d45 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.613143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.629350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34829137-c36b-44fa-b8c0-5fe5201c5d45 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.663085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f45652-864f-435d-8987-a6b7fc487c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.663751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f45652-864f-435d-8987-a6b7fc487c03 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.664026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f45652-864f-435d-8987-a6b7fc487c03 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.664294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f45652-864f-435d-8987-a6b7fc487c03 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.665010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f45652-864f-435d-8987-a6b7fc487c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:28.665309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f45652-864f-435d-8987-a6b7fc487c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.667217 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f45652-864f-435d-8987-a6b7fc487c03 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.682503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 504/1022] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:28.686648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680bb71-c774-4b7a-b39f-1cf8211a6ee0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.788460 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680bb71-c774-4b7a-b39f-1cf8211a6ee0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.822431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e5708-53fc-4987-8a59-869a61a726aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.823243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e5708-53fc-4987-8a59-869a61a726aa None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.823521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e5708-53fc-4987-8a59-869a61a726aa None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.823793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e5708-53fc-4987-8a59-869a61a726aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.830867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e5708-53fc-4987-8a59-869a61a726aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.832303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e5708-53fc-4987-8a59-869a61a726aa None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-fd7e5708-53fc-4987-8a59-869a61a726aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:28.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:28.853811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:28.854860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 519/1023] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:28.855524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f45652-864f-435d-8987-a6b7fc487c03 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.960561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f45652-864f-435d-8987-a6b7fc487c03 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.994622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb36779-6620-4415-9a7d-e966f523d77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.995288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36779-6620-4415-9a7d-e966f523d77c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.995563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36779-6620-4415-9a7d-e966f523d77c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.995833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36779-6620-4415-9a7d-e966f523d77c None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.996648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36779-6620-4415-9a7d-e966f523d77c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:28.996948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36779-6620-4415-9a7d-e966f523d77c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.998352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36779-6620-4415-9a7d-e966f523d77c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.020464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 505/1024] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:29.021871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e5708-53fc-4987-8a59-869a61a726aa None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.070853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.121229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e5708-53fc-4987-8a59-869a61a726aa None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.154208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a2f85-8cf7-4a3c-8297-6bf66bcdf8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.154875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a2f85-8cf7-4a3c-8297-6bf66bcdf8fb None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.155155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a2f85-8cf7-4a3c-8297-6bf66bcdf8fb None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.155427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a2f85-8cf7-4a3c-8297-6bf66bcdf8fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.169577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a8acf-f7f5-4fe7-9da0-fa7bcfe284f9 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.169840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a8acf-f7f5-4fe7-9da0-fa7bcfe284f9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.170071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a8acf-f7f5-4fe7-9da0-fa7bcfe284f9 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.171628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 506/1025] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:29.176235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49fee9-02c9-45c5-811c-14e51cdcde3e None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.176516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49fee9-02c9-45c5-811c-14e51cdcde3e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.176861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49fee9-02c9-45c5-811c-14e51cdcde3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.182405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.194280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3f49fee9-02c9-45c5-811c-14e51cdcde3e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:29.195863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3f49fee9-02c9-45c5-811c-14e51cdcde3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:29.226062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a2f85-8cf7-4a3c-8297-6bf66bcdf8fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.227559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a2f85-8cf7-4a3c-8297-6bf66bcdf8fb None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.265115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 507/1026] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:29.280504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.287613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49fee9-02c9-45c5-811c-14e51cdcde3e None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.338916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 520/1027] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:41:29.340136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a2f85-8cf7-4a3c-8297-6bf66bcdf8fb None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.396708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49fee9-02c9-45c5-811c-14e51cdcde3e None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.426769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.433439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2e177-58d6-4438-a668-8fd4e37ef1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.434229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2e177-58d6-4438-a668-8fd4e37ef1ca None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.434533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2e177-58d6-4438-a668-8fd4e37ef1ca None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.438766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2e177-58d6-4438-a668-8fd4e37ef1ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.438766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2e177-58d6-4438-a668-8fd4e37ef1ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.438766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2e177-58d6-4438-a668-8fd4e37ef1ca None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.439015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 508/1028] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:29.441361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a2f85-8cf7-4a3c-8297-6bf66bcdf8fb None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.444563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2e177-58d6-4438-a668-8fd4e37ef1ca None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.476928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02582d19-bf4d-4f3b-92a1-5b23df95c171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.477727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02582d19-bf4d-4f3b-92a1-5b23df95c171 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.478564 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02582d19-bf4d-4f3b-92a1-5b23df95c171 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.478841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02582d19-bf4d-4f3b-92a1-5b23df95c171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.489064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02582d19-bf4d-4f3b-92a1-5b23df95c171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.490541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02582d19-bf4d-4f3b-92a1-5b23df95c171 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.503247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-02582d19-bf4d-4f3b-92a1-5b23df95c171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:29.512662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:29.513755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:29.515110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:29.515110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:29.515110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:29.515110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:29.515110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 521/1029] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:29.519953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02582d19-bf4d-4f3b-92a1-5b23df95c171 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.554130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2e177-58d6-4438-a668-8fd4e37ef1ca None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.587197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d7660-f58e-4302-ba7b-810e2f1d40ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.588371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d7660-f58e-4302-ba7b-810e2f1d40ab None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.588647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d7660-f58e-4302-ba7b-810e2f1d40ab None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.588919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d7660-f58e-4302-ba7b-810e2f1d40ab None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.589612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d7660-f58e-4302-ba7b-810e2f1d40ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.591068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d7660-f58e-4302-ba7b-810e2f1d40ab None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.618966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02582d19-bf4d-4f3b-92a1-5b23df95c171 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.652360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316f0ed-c506-46e5-a2ea-22732f9920d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.653019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316f0ed-c506-46e5-a2ea-22732f9920d5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.653299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316f0ed-c506-46e5-a2ea-22732f9920d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.653572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316f0ed-c506-46e5-a2ea-22732f9920d5 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.654292 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316f0ed-c506-46e5-a2ea-22732f9920d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:29.654632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316f0ed-c506-46e5-a2ea-22732f9920d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.655993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316f0ed-c506-46e5-a2ea-22732f9920d5 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.667939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 522/1030] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:29.673360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316f0ed-c506-46e5-a2ea-22732f9920d5 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.705631 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 509/1031] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:41:29 2022] POST /identity/v3/projects => generated 311 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:41:29.739115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.785416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316f0ed-c506-46e5-a2ea-22732f9920d5 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.821982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810645e-085b-4f13-adc6-f372d9289d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.822833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810645e-085b-4f13-adc6-f372d9289d7d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.823116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810645e-085b-4f13-adc6-f372d9289d7d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.823391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810645e-085b-4f13-adc6-f372d9289d7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.831419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810645e-085b-4f13-adc6-f372d9289d7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.832888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810645e-085b-4f13-adc6-f372d9289d7d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-9810645e-085b-4f13-adc6-f372d9289d7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:29.848883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:29.851033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:29.851980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:29.851980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:29.851980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:29.851980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:29.851980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:29.851980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:29.851980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:29.851980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:29.851980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 523/1032] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:29.856584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d7660-f58e-4302-ba7b-810e2f1d40ab None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.920349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.970277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d7660-f58e-4302-ba7b-810e2f1d40ab None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.003555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f24d12-071f-482a-b9d5-44a4ed797396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.004222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f24d12-071f-482a-b9d5-44a4ed797396 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.004502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f24d12-071f-482a-b9d5-44a4ed797396 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.004776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f24d12-071f-482a-b9d5-44a4ed797396 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.005548 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f24d12-071f-482a-b9d5-44a4ed797396 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:30.005852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f24d12-071f-482a-b9d5-44a4ed797396 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.007299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f24d12-071f-482a-b9d5-44a4ed797396 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.020324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 510/1033] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:30.026724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810645e-085b-4f13-adc6-f372d9289d7d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.126027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810645e-085b-4f13-adc6-f372d9289d7d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.146613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.160949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba530f3e-d467-4863-88d4-5e2575c540e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.161609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba530f3e-d467-4863-88d4-5e2575c540e7 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.161886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba530f3e-d467-4863-88d4-5e2575c540e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.162161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba530f3e-d467-4863-88d4-5e2575c540e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.163026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba530f3e-d467-4863-88d4-5e2575c540e7 None admin] RBAC: Adding query filter params (role.id=333abb19b7ef46c7a17f4f86c8d798e3, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=067b827625e0427294b93b027597a5f3) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:30.163338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba530f3e-d467-4863-88d4-5e2575c540e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.164772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba530f3e-d467-4863-88d4-5e2575c540e7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.174500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 524/1034] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/role_assignments?user.id=067b827625e0427294b93b027597a5f3&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=333abb19b7ef46c7a17f4f86c8d798e3 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:30.274351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb10989-3311-4ca4-8ad3-be23f890bb8f None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.274688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb10989-3311-4ca4-8ad3-be23f890bb8f None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.275463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb10989-3311-4ca4-8ad3-be23f890bb8f None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.277064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 511/1035] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:30.282678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6561d3-b94a-42b8-9245-e9ab7c814910 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.282982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6561d3-b94a-42b8-9245-e9ab7c814910 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.283238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6561d3-b94a-42b8-9245-e9ab7c814910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.300541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-8a6561d3-b94a-42b8-9245-e9ab7c814910 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:30.301755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8a6561d3-b94a-42b8-9245-e9ab7c814910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:30.374843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 525/1036] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:30.397967 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb10989-3311-4ca4-8ad3-be23f890bb8f None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.505861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb10989-3311-4ca4-8ad3-be23f890bb8f None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.519913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.541969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc882671-87c3-4ace-8bfe-85a74922adeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.542709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc882671-87c3-4ace-8bfe-85a74922adeb None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.542987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc882671-87c3-4ace-8bfe-85a74922adeb None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.543258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc882671-87c3-4ace-8bfe-85a74922adeb None admin] PATH_INFO: `/v3/services/image` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.543901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc882671-87c3-4ace-8bfe-85a74922adeb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.545251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc882671-87c3-4ace-8bfe-85a74922adeb None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-bc882671-87c3-4ace-8bfe-85a74922adeb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:30.556856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:30.557869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:41:30.559161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:30.562557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 512/1037] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:30.564221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6561d3-b94a-42b8-9245-e9ab7c814910 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.665150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6561d3-b94a-42b8-9245-e9ab7c814910 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.678906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.699256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bb721-b536-43ca-982c-a4017e44c651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.699920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bb721-b536-43ca-982c-a4017e44c651 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.700207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bb721-b536-43ca-982c-a4017e44c651 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.700487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bb721-b536-43ca-982c-a4017e44c651 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.701193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5bb721-b536-43ca-982c-a4017e44c651 None admin] RBAC: Adding query filter params (name=image) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:30.701497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5bb721-b536-43ca-982c-a4017e44c651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.702907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5bb721-b536-43ca-982c-a4017e44c651 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.712869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 526/1038] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:30.714415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc882671-87c3-4ace-8bfe-85a74922adeb None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.817318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc882671-87c3-4ace-8bfe-85a74922adeb None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.852703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48617-8e9d-4085-bd67-07b108d83346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.853787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48617-8e9d-4085-bd67-07b108d83346 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.854323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48617-8e9d-4085-bd67-07b108d83346 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.854835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48617-8e9d-4085-bd67-07b108d83346 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.855766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff48617-8e9d-4085-bd67-07b108d83346 None admin] RBAC: Adding query filter params (type=image) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:30.856318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff48617-8e9d-4085-bd67-07b108d83346 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.857881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff48617-8e9d-4085-bd67-07b108d83346 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.868438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 513/1039] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:30.869702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bb721-b536-43ca-982c-a4017e44c651 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.978574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bb721-b536-43ca-982c-a4017e44c651 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:31.011933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f165f-ff29-46a3-844c-41a9cdfd484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:31.012581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f165f-ff29-46a3-844c-41a9cdfd484d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.012862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f165f-ff29-46a3-844c-41a9cdfd484d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.013135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f165f-ff29-46a3-844c-41a9cdfd484d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.014174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f165f-ff29-46a3-844c-41a9cdfd484d None admin] RBAC: Adding query filter params (interface=public, service_id=c693116c01f94310a9e113ff330b5a17, region_id=RegionOne) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:31.014506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f165f-ff29-46a3-844c-41a9cdfd484d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.015878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f165f-ff29-46a3-844c-41a9cdfd484d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:31.026025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 527/1040] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/endpoints?service_id=c693116c01f94310a9e113ff330b5a17&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:31.027300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48617-8e9d-4085-bd67-07b108d83346 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:31.073222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:31.129946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48617-8e9d-4085-bd67-07b108d83346 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:31.163489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3e490-806d-497c-b7e3-fa94ee776924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:31.164155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3e490-806d-497c-b7e3-fa94ee776924 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.164442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3e490-806d-497c-b7e3-fa94ee776924 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.164718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3e490-806d-497c-b7e3-fa94ee776924 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.165432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3e490-806d-497c-b7e3-fa94ee776924 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.166824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3e490-806d-497c-b7e3-fa94ee776924 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:31.173532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 514/1041] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 05:41:31 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:41:31.817242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029262de-8c08-4a17-a32e-f3f917e271b6 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.817529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029262de-8c08-4a17-a32e-f3f917e271b6 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.817810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029262de-8c08-4a17-a32e-f3f917e271b6 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.819250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 528/1042] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:31.824775 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1977ea-c0f3-4887-aae2-97bad43d09c5 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.825068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1977ea-c0f3-4887-aae2-97bad43d09c5 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.825350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1977ea-c0f3-4887-aae2-97bad43d09c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.842576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8a1977ea-c0f3-4887-aae2-97bad43d09c5 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:31.843775 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8a1977ea-c0f3-4887-aae2-97bad43d09c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:31.920873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 515/1043] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:31.945305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029262de-8c08-4a17-a32e-f3f917e271b6 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:31.991471 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.050227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029262de-8c08-4a17-a32e-f3f917e271b6 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.083227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87690e8-0817-411e-beee-9cf031db12ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.083948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87690e8-0817-411e-beee-9cf031db12ad None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.084233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87690e8-0817-411e-beee-9cf031db12ad None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.084509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87690e8-0817-411e-beee-9cf031db12ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.085681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87690e8-0817-411e-beee-9cf031db12ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.087677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87690e8-0817-411e-beee-9cf031db12ad None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.094063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 529/1044] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:32.095306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1977ea-c0f3-4887-aae2-97bad43d09c5 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.130441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.171895 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bb433-b681-4008-a91c-a9b2e2f1c176 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.172228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bb433-b681-4008-a91c-a9b2e2f1c176 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.172468 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bb433-b681-4008-a91c-a9b2e2f1c176 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.174439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 530/1045] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:32.179923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135ddf1-19a1-4fff-99e0-56447f857d4c None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.180216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135ddf1-19a1-4fff-99e0-56447f857d4c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.180504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135ddf1-19a1-4fff-99e0-56447f857d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.197581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1135ddf1-19a1-4fff-99e0-56447f857d4c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:32.198804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1135ddf1-19a1-4fff-99e0-56447f857d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:32.204921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1977ea-c0f3-4887-aae2-97bad43d09c5 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.241296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd031f23-051d-4648-abd0-08ae2321847f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.241975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd031f23-051d-4648-abd0-08ae2321847f None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.242258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd031f23-051d-4648-abd0-08ae2321847f None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.242567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd031f23-051d-4648-abd0-08ae2321847f None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.243283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd031f23-051d-4648-abd0-08ae2321847f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.244758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd031f23-051d-4648-abd0-08ae2321847f None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.271681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 531/1046] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:32.295813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135ddf1-19a1-4fff-99e0-56447f857d4c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.338583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 516/1047] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:41:32 2022] POST /identity/v3/users => generated 285 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:32.341945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.403695 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135ddf1-19a1-4fff-99e0-56447f857d4c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.439143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc7335-621a-4e08-b895-1b13af94ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.439975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc7335-621a-4e08-b895-1b13af94ac56 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.440254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc7335-621a-4e08-b895-1b13af94ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.440532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc7335-621a-4e08-b895-1b13af94ac56 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.452503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc7335-621a-4e08-b895-1b13af94ac56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-eafc7335-621a-4e08-b895-1b13af94ac56 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:32.460980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:32.462032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:32.463490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 532/1048] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:32.469437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd031f23-051d-4648-abd0-08ae2321847f None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.568699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.579236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd031f23-051d-4648-abd0-08ae2321847f None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.613880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a29b2-c70c-47aa-8378-9c060fd9b07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.614947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a29b2-c70c-47aa-8378-9c060fd9b07b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.615421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a29b2-c70c-47aa-8378-9c060fd9b07b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.621098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a29b2-c70c-47aa-8378-9c060fd9b07b None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.621098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a29b2-c70c-47aa-8378-9c060fd9b07b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:32.621098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a29b2-c70c-47aa-8378-9c060fd9b07b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.621098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a29b2-c70c-47aa-8378-9c060fd9b07b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.627966 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 517/1049] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:32.633609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc7335-621a-4e08-b895-1b13af94ac56 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.740065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.744206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc7335-621a-4e08-b895-1b13af94ac56 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.778102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc1e91-4968-4d4a-abab-79acf3cfc880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.778951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc1e91-4968-4d4a-abab-79acf3cfc880 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.779620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc1e91-4968-4d4a-abab-79acf3cfc880 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.780295 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc1e91-4968-4d4a-abab-79acf3cfc880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.791160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc1e91-4968-4d4a-abab-79acf3cfc880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.792624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc1e91-4968-4d4a-abab-79acf3cfc880 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-9bdc1e91-4968-4d4a-abab-79acf3cfc880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:32.810685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:32.811599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:32.812498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:32.812498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:32.812498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:32.812498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:32.816769 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 533/1050] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:32.818109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a29b2-c70c-47aa-8378-9c060fd9b07b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.924713 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a29b2-c70c-47aa-8378-9c060fd9b07b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.959483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0ce58-27a8-4448-ad0e-03534fd9a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.960149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0ce58-27a8-4448-ad0e-03534fd9a52c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.960426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0ce58-27a8-4448-ad0e-03534fd9a52c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.960699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0ce58-27a8-4448-ad0e-03534fd9a52c None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.961408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0ce58-27a8-4448-ad0e-03534fd9a52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:32.961710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0ce58-27a8-4448-ad0e-03534fd9a52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.963226 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0ce58-27a8-4448-ad0e-03534fd9a52c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.979616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 518/1051] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:32.981052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc1e91-4968-4d4a-abab-79acf3cfc880 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.082038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc1e91-4968-4d4a-abab-79acf3cfc880 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.115799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb097b-9acf-4a08-adde-12b7210e53a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.116592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb097b-9acf-4a08-adde-12b7210e53a6 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.116869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb097b-9acf-4a08-adde-12b7210e53a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.117137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb097b-9acf-4a08-adde-12b7210e53a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.125719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb097b-9acf-4a08-adde-12b7210e53a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.127229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb097b-9acf-4a08-adde-12b7210e53a6 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-febb097b-9acf-4a08-adde-12b7210e53a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:33.143862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:33.144883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:33.145862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 534/1052] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:33.152413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0ce58-27a8-4448-ad0e-03534fd9a52c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.168269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.253087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0ce58-27a8-4448-ad0e-03534fd9a52c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.282772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767b9b-e8bc-497e-aa6f-d17d586f7d95 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.283134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767b9b-e8bc-497e-aa6f-d17d586f7d95 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.283487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767b9b-e8bc-497e-aa6f-d17d586f7d95 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.285184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 535/1053] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:33.288501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984bc62-d882-46b8-a9b7-ac6a2eb049ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.289200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984bc62-d882-46b8-a9b7-ac6a2eb049ce None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.289480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984bc62-d882-46b8-a9b7-ac6a2eb049ce None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.289724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10ebae-c758-44ac-a998-1b6dc42430fe None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.289835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984bc62-d882-46b8-a9b7-ac6a2eb049ce None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.290002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10ebae-c758-44ac-a998-1b6dc42430fe None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.290918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10ebae-c758-44ac-a998-1b6dc42430fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.291134 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984bc62-d882-46b8-a9b7-ac6a2eb049ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:33.291440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984bc62-d882-46b8-a9b7-ac6a2eb049ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.292845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984bc62-d882-46b8-a9b7-ac6a2eb049ce None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.307245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-2e10ebae-c758-44ac-a998-1b6dc42430fe None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:33.308453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2e10ebae-c758-44ac-a998-1b6dc42430fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:33.311018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 519/1054] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:33.317225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984bc62-d882-46b8-a9b7-ac6a2eb049ce None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.320866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.390511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 536/1055] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:33.394987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.410706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10ebae-c758-44ac-a998-1b6dc42430fe None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.433227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984bc62-d882-46b8-a9b7-ac6a2eb049ce None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.482609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef38c0-1de1-45db-a858-48203c6ca0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.483845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef38c0-1de1-45db-a858-48203c6ca0e0 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.484274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef38c0-1de1-45db-a858-48203c6ca0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.485140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef38c0-1de1-45db-a858-48203c6ca0e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.498574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef38c0-1de1-45db-a858-48203c6ca0e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.500247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef38c0-1de1-45db-a858-48203c6ca0e0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-6bef38c0-1de1-45db-a858-48203c6ca0e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:33.518300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:33.519455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:33.520482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:33.520482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:33.520482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:33.520482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:33.520482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 520/1056] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:33.520767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10ebae-c758-44ac-a998-1b6dc42430fe None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.525568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef38c0-1de1-45db-a858-48203c6ca0e0 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.557627 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe373b-b0bd-484a-9211-494ddf4a640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.558417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe373b-b0bd-484a-9211-494ddf4a640b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.558723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe373b-b0bd-484a-9211-494ddf4a640b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.558995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe373b-b0bd-484a-9211-494ddf4a640b None admin] PATH_INFO: `/v3/services/image` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.559653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe373b-b0bd-484a-9211-494ddf4a640b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.561020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe373b-b0bd-484a-9211-494ddf4a640b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-88fe373b-b0bd-484a-9211-494ddf4a640b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:33.572892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:33.573920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:33.574803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 537/1057] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:33.579809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe373b-b0bd-484a-9211-494ddf4a640b None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.589458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.636564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef38c0-1de1-45db-a858-48203c6ca0e0 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.661434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.675515 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f27de-b924-4241-86a2-b088fae38236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.676193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f27de-b924-4241-86a2-b088fae38236 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.676466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f27de-b924-4241-86a2-b088fae38236 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.676731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f27de-b924-4241-86a2-b088fae38236 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.677447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f27de-b924-4241-86a2-b088fae38236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:33.677751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f27de-b924-4241-86a2-b088fae38236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.679170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f27de-b924-4241-86a2-b088fae38236 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.682239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe373b-b0bd-484a-9211-494ddf4a640b None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.690896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 521/1058] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:33.696154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f27de-b924-4241-86a2-b088fae38236 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.716467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbc2dc-ea1e-4365-82db-851865b6f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.717127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbc2dc-ea1e-4365-82db-851865b6f63a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.717410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbc2dc-ea1e-4365-82db-851865b6f63a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.717676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbc2dc-ea1e-4365-82db-851865b6f63a None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.718382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbc2dc-ea1e-4365-82db-851865b6f63a None admin] RBAC: Adding query filter params (name=image) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:33.718707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbc2dc-ea1e-4365-82db-851865b6f63a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.720070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbc2dc-ea1e-4365-82db-851865b6f63a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.725007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 538/1059] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:33.730911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbc2dc-ea1e-4365-82db-851865b6f63a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.812958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.816738 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f27de-b924-4241-86a2-b088fae38236 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.844411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbc2dc-ea1e-4365-82db-851865b6f63a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.855229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e4a08-2276-4cd7-86f7-e7cb77d4fdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.856033 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e4a08-2276-4cd7-86f7-e7cb77d4fdd5 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.856314 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e4a08-2276-4cd7-86f7-e7cb77d4fdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.856587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e4a08-2276-4cd7-86f7-e7cb77d4fdd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.863212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e4a08-2276-4cd7-86f7-e7cb77d4fdd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.864642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e4a08-2276-4cd7-86f7-e7cb77d4fdd5 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-ea8e4a08-2276-4cd7-86f7-e7cb77d4fdd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:33.879410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:33.880315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:33.881248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:33.881248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:33.881248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:33.881248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:33.881248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:33.881248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:33.881248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:33.881248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:33.881692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74891ed8-ee8f-4911-98f9-b3b9376b4a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.881692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74891ed8-ee8f-4911-98f9-b3b9376b4a3d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.881848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74891ed8-ee8f-4911-98f9-b3b9376b4a3d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.882406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74891ed8-ee8f-4911-98f9-b3b9376b4a3d None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.882747 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74891ed8-ee8f-4911-98f9-b3b9376b4a3d None admin] RBAC: Adding query filter params (type=image) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:33.883051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74891ed8-ee8f-4911-98f9-b3b9376b4a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.884445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74891ed8-ee8f-4911-98f9-b3b9376b4a3d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.885567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 522/1060] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:33.886204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e4a08-2276-4cd7-86f7-e7cb77d4fdd5 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.895075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 539/1061] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:33.895953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74891ed8-ee8f-4911-98f9-b3b9376b4a3d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.939599 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.996098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e4a08-2276-4cd7-86f7-e7cb77d4fdd5 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.005735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74891ed8-ee8f-4911-98f9-b3b9376b4a3d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.031142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26785bdd-38f4-446b-a441-1dda580234ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.031804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26785bdd-38f4-446b-a441-1dda580234ba None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.032087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26785bdd-38f4-446b-a441-1dda580234ba None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.032360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26785bdd-38f4-446b-a441-1dda580234ba None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.033134 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26785bdd-38f4-446b-a441-1dda580234ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:34.033438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26785bdd-38f4-446b-a441-1dda580234ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.034933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26785bdd-38f4-446b-a441-1dda580234ba None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.044196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420aece-b80c-4797-a572-9c2aed9c0514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.045051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420aece-b80c-4797-a572-9c2aed9c0514 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.045514 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420aece-b80c-4797-a572-9c2aed9c0514 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.045940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420aece-b80c-4797-a572-9c2aed9c0514 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.046861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 523/1062] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:34.047066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420aece-b80c-4797-a572-9c2aed9c0514 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.048620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420aece-b80c-4797-a572-9c2aed9c0514 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.052215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:34.052587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26785bdd-38f4-446b-a441-1dda580234ba None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.122687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 540/1063] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 05:41:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:34.157679 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26785bdd-38f4-446b-a441-1dda580234ba None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.159401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:34.203734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f20af-05e3-413f-8d72-840995e901b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.204655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f20af-05e3-413f-8d72-840995e901b4 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.204933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f20af-05e3-413f-8d72-840995e901b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.205207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f20af-05e3-413f-8d72-840995e901b4 None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/067b827625e0427294b93b027597a5f3/roles/333abb19b7ef46c7a17f4f86c8d798e3` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.207501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f20af-05e3-413f-8d72-840995e901b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=067b827625e0427294b93b027597a5f3, role_id=333abb19b7ef46c7a17f4f86c8d798e3)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.208942 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f20af-05e3-413f-8d72-840995e901b4 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.272152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 524/1064] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:34 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/067b827625e0427294b93b027597a5f3/roles/333abb19b7ef46c7a17f4f86c8d798e3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:41:34.495932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb09e6b-1e54-46a7-979f-113262141aba None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.496300 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb09e6b-1e54-46a7-979f-113262141aba None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.496487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb09e6b-1e54-46a7-979f-113262141aba None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.497914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 541/1065] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:34.503266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686a20a-7b2c-407f-8db8-c9f32618e9e8 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.503613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686a20a-7b2c-407f-8db8-c9f32618e9e8 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.505404 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686a20a-7b2c-407f-8db8-c9f32618e9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.521572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-d686a20a-7b2c-407f-8db8-c9f32618e9e8 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:34.522851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d686a20a-7b2c-407f-8db8-c9f32618e9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:34.589415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 525/1066] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:34.612956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb09e6b-1e54-46a7-979f-113262141aba None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.715553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb09e6b-1e54-46a7-979f-113262141aba None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.753135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa7b74-d336-4a0c-b551-a2cdeace3eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.753932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa7b74-d336-4a0c-b551-a2cdeace3eec None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.754216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa7b74-d336-4a0c-b551-a2cdeace3eec None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.754518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa7b74-d336-4a0c-b551-a2cdeace3eec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.765181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa7b74-d336-4a0c-b551-a2cdeace3eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5daa7b74-d336-4a0c-b551-a2cdeace3eec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:34.773504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:34.775229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:34.776141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:34.778800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 542/1067] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:34.781388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686a20a-7b2c-407f-8db8-c9f32618e9e8 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.882768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:34.898587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686a20a-7b2c-407f-8db8-c9f32618e9e8 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.935561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d45b8-3356-4c09-9e94-ee61b4fa37dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.936235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d45b8-3356-4c09-9e94-ee61b4fa37dd None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.936518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d45b8-3356-4c09-9e94-ee61b4fa37dd None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.936795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d45b8-3356-4c09-9e94-ee61b4fa37dd None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.937539 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d45b8-3356-4c09-9e94-ee61b4fa37dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:34.937903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d45b8-3356-4c09-9e94-ee61b4fa37dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.939291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d45b8-3356-4c09-9e94-ee61b4fa37dd None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.952706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 526/1068] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:34.953905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa7b74-d336-4a0c-b551-a2cdeace3eec None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.005724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:35.056550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa7b74-d336-4a0c-b551-a2cdeace3eec None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.091350 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3177aa-2811-440a-ad32-620f27af276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.092224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3177aa-2811-440a-ad32-620f27af276e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.092505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3177aa-2811-440a-ad32-620f27af276e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.092776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3177aa-2811-440a-ad32-620f27af276e None admin] PATH_INFO: `/v3/users/ac66985266e8416b81dcfd913af3c165` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.105908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3177aa-2811-440a-ad32-620f27af276e None admin] RBAC: Authorizing `identity:get_user(user_id=ac66985266e8416b81dcfd913af3c165)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.107381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3177aa-2811-440a-ad32-620f27af276e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.113688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 543/1069] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:41:34 2022] GET /identity/v3/users/ac66985266e8416b81dcfd913af3c165 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:35.114795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d45b8-3356-4c09-9e94-ee61b4fa37dd None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.213785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d45b8-3356-4c09-9e94-ee61b4fa37dd None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.248560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db6130-7e18-4cb0-a729-a7703d1ac3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.249217 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db6130-7e18-4cb0-a729-a7703d1ac3fa None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.249505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db6130-7e18-4cb0-a729-a7703d1ac3fa None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.249782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db6130-7e18-4cb0-a729-a7703d1ac3fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.269052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:35.321058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db6130-7e18-4cb0-a729-a7703d1ac3fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.322494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db6130-7e18-4cb0-a729-a7703d1ac3fa None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.428085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 527/1070] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:35.429288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3177aa-2811-440a-ad32-620f27af276e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.534678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3177aa-2811-440a-ad32-620f27af276e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.543778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:35.568442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db08e8-04a9-49ac-9edf-96451e03f401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.569299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db08e8-04a9-49ac-9edf-96451e03f401 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.569582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db08e8-04a9-49ac-9edf-96451e03f401 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.569860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db08e8-04a9-49ac-9edf-96451e03f401 None admin] PATH_INFO: `/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.571108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db08e8-04a9-49ac-9edf-96451e03f401 None admin] RBAC: Authorizing `identity:get_project(project_id=1a0f2a9e182d4ef494ab1a2bae7c2c63)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.573374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db08e8-04a9-49ac-9edf-96451e03f401 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.581203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db6130-7e18-4cb0-a729-a7703d1ac3fa None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.582551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 544/1071] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:35.630520 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:35.683875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db6130-7e18-4cb0-a729-a7703d1ac3fa None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.718068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211114-6be1-4cb2-a8d4-3bd40adadf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.718762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211114-6be1-4cb2-a8d4-3bd40adadf56 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.719047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211114-6be1-4cb2-a8d4-3bd40adadf56 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.719325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211114-6be1-4cb2-a8d4-3bd40adadf56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.720161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211114-6be1-4cb2-a8d4-3bd40adadf56 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=1a0f2a9e182d4ef494ab1a2bae7c2c63, user.id=ac66985266e8416b81dcfd913af3c165) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:35.720477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211114-6be1-4cb2-a8d4-3bd40adadf56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.721906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211114-6be1-4cb2-a8d4-3bd40adadf56 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.733900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 528/1072] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/role_assignments?user.id=ac66985266e8416b81dcfd913af3c165&scope.project.id=1a0f2a9e182d4ef494ab1a2bae7c2c63&role.id=7a833c45603a4830b888e58e1232712a => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:36.197835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85f4ee-4c6a-4bd7-b7f2-4f76eeb70d0a None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.198343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85f4ee-4c6a-4bd7-b7f2-4f76eeb70d0a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.198343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85f4ee-4c6a-4bd7-b7f2-4f76eeb70d0a None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.199760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 545/1073] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:36.205000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb910f16-5830-4529-9741-ea42c8992e48 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.205404 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb910f16-5830-4529-9741-ea42c8992e48 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.205585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb910f16-5830-4529-9741-ea42c8992e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.221992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-bb910f16-5830-4529-9741-ea42c8992e48 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:36.223205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bb910f16-5830-4529-9741-ea42c8992e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:36.289920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 529/1074] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:36.295481 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff94932-c7e0-419f-b4aa-caea70099a74 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.295770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff94932-c7e0-419f-b4aa-caea70099a74 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.296046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff94932-c7e0-419f-b4aa-caea70099a74 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.297533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 546/1075] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:36.302296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16da77-040b-4d1e-86d7-9a980a36172e None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.302616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16da77-040b-4d1e-86d7-9a980a36172e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.302895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16da77-040b-4d1e-86d7-9a980a36172e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.312486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff94932-c7e0-419f-b4aa-caea70099a74 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.319205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3d16da77-040b-4d1e-86d7-9a980a36172e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:36.320380 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3d16da77-040b-4d1e-86d7-9a980a36172e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:36.366760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.396373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 530/1076] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:36.417778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff94932-c7e0-419f-b4aa-caea70099a74 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.420567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16da77-040b-4d1e-86d7-9a980a36172e None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.452531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab3b40-ca8d-4e3b-9819-382f0c411998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.453461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab3b40-ca8d-4e3b-9819-382f0c411998 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.453936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab3b40-ca8d-4e3b-9819-382f0c411998 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.454208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab3b40-ca8d-4e3b-9819-382f0c411998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.500718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.520702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16da77-040b-4d1e-86d7-9a980a36172e None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.524194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab3b40-ca8d-4e3b-9819-382f0c411998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:36.526191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab3b40-ca8d-4e3b-9819-382f0c411998 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:36.555172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a299c53-ead7-4e79-bc45-598398466eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.556026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a299c53-ead7-4e79-bc45-598398466eea None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.556289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a299c53-ead7-4e79-bc45-598398466eea None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.556561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a299c53-ead7-4e79-bc45-598398466eea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.566495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a299c53-ead7-4e79-bc45-598398466eea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-8a299c53-ead7-4e79-bc45-598398466eea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:36.574592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:36.575531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:36.576393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 531/1077] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:36.581622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a299c53-ead7-4e79-bc45-598398466eea None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.615311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.632034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 547/1078] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:41:36.638210 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab3b40-ca8d-4e3b-9819-382f0c411998 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.684281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a299c53-ead7-4e79-bc45-598398466eea None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.718128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f9662-5a1c-49c1-8a34-c2413863be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.718752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9662-5a1c-49c1-8a34-c2413863be09 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.719034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9662-5a1c-49c1-8a34-c2413863be09 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.719306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9662-5a1c-49c1-8a34-c2413863be09 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.720045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f9662-5a1c-49c1-8a34-c2413863be09 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:36.720353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f9662-5a1c-49c1-8a34-c2413863be09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:36.721716 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f9662-5a1c-49c1-8a34-c2413863be09 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:36.722612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.735381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 532/1079] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:36.738442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f9662-5a1c-49c1-8a34-c2413863be09 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.751933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab3b40-ca8d-4e3b-9819-382f0c411998 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.791225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7c8df-d51a-4d8c-a2be-1e1d9facc528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.792026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7c8df-d51a-4d8c-a2be-1e1d9facc528 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.792307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7c8df-d51a-4d8c-a2be-1e1d9facc528 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.792580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7c8df-d51a-4d8c-a2be-1e1d9facc528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.803859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7c8df-d51a-4d8c-a2be-1e1d9facc528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:36.805354 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7c8df-d51a-4d8c-a2be-1e1d9facc528 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-1ed7c8df-d51a-4d8c-a2be-1e1d9facc528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:36.832319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:36.833307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:36.834198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:36.834198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:36.834198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:36.834198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:36.834198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 548/1080] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:36.839866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7c8df-d51a-4d8c-a2be-1e1d9facc528 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.860228 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.866651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f9662-5a1c-49c1-8a34-c2413863be09 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.907005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a346a-4aec-4846-a1d5-f643a009d45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.907806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a346a-4aec-4846-a1d5-f643a009d45c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.908085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a346a-4aec-4846-a1d5-f643a009d45c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.908359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a346a-4aec-4846-a1d5-f643a009d45c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.919284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a346a-4aec-4846-a1d5-f643a009d45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:36.920863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a346a-4aec-4846-a1d5-f643a009d45c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-3d8a346a-4aec-4846-a1d5-f643a009d45c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:36.943865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:36.944800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:36.945746 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:36.945746 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:36.945746 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:36.945746 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:36.951209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 533/1081] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:36.952299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a346a-4aec-4846-a1d5-f643a009d45c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.959187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7c8df-d51a-4d8c-a2be-1e1d9facc528 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.987778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.996159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf077c-af7b-422a-b47f-e6dda0386641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.996859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf077c-af7b-422a-b47f-e6dda0386641 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.997169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf077c-af7b-422a-b47f-e6dda0386641 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.997443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf077c-af7b-422a-b47f-e6dda0386641 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.998228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf077c-af7b-422a-b47f-e6dda0386641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:36.998565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf077c-af7b-422a-b47f-e6dda0386641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.005250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf077c-af7b-422a-b47f-e6dda0386641 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.012814 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 549/1082] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:37.065864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a346a-4aec-4846-a1d5-f643a009d45c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.106425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a7ac-a676-4072-89c4-062f5f6ceffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.107138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a7ac-a676-4072-89c4-062f5f6ceffe None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.107490 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a7ac-a676-4072-89c4-062f5f6ceffe None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.107708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a7ac-a676-4072-89c4-062f5f6ceffe None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.108449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03a7ac-a676-4072-89c4-062f5f6ceffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:37.108761 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03a7ac-a676-4072-89c4-062f5f6ceffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.110169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03a7ac-a676-4072-89c4-062f5f6ceffe None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.123146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 534/1083] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:37.128746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf077c-af7b-422a-b47f-e6dda0386641 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.249124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf077c-af7b-422a-b47f-e6dda0386641 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.285572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c85dbe-7499-4cc3-84a4-2ed0dbfee220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.286533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c85dbe-7499-4cc3-84a4-2ed0dbfee220 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.288180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c85dbe-7499-4cc3-84a4-2ed0dbfee220 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.288180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c85dbe-7499-4cc3-84a4-2ed0dbfee220 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.296337 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c85dbe-7499-4cc3-84a4-2ed0dbfee220 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.297804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c85dbe-7499-4cc3-84a4-2ed0dbfee220 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-09c85dbe-7499-4cc3-84a4-2ed0dbfee220 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:37.312956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:37.313981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:37.315038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 550/1084] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:37.322946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a7ac-a676-4072-89c4-062f5f6ceffe None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.335267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.428459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a7ac-a676-4072-89c4-062f5f6ceffe None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.461819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c6bcd-0ffb-4ec4-8b03-036cf061b92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.463097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c6bcd-0ffb-4ec4-8b03-036cf061b92c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.463374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c6bcd-0ffb-4ec4-8b03-036cf061b92c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.463645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c6bcd-0ffb-4ec4-8b03-036cf061b92c None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.464474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c6bcd-0ffb-4ec4-8b03-036cf061b92c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:37.464776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c6bcd-0ffb-4ec4-8b03-036cf061b92c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.466191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c6bcd-0ffb-4ec4-8b03-036cf061b92c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.483557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 535/1085] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:37.489319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c85dbe-7499-4cc3-84a4-2ed0dbfee220 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.540753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.595906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c85dbe-7499-4cc3-84a4-2ed0dbfee220 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.629175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db8eb1-8cc4-4bad-8303-e77809cacc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.629818 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db8eb1-8cc4-4bad-8303-e77809cacc56 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.630096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db8eb1-8cc4-4bad-8303-e77809cacc56 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.630370 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db8eb1-8cc4-4bad-8303-e77809cacc56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.699957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db8eb1-8cc4-4bad-8303-e77809cacc56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.701409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db8eb1-8cc4-4bad-8303-e77809cacc56 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.749392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.779721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3322304-4d97-4873-9655-b6211feb9bdc None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.779997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3322304-4d97-4873-9655-b6211feb9bdc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.780272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3322304-4d97-4873-9655-b6211feb9bdc None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.781652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 536/1086] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:37.786808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da22ea-c980-4a4f-8ed4-e40293d42951 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.787092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da22ea-c980-4a4f-8ed4-e40293d42951 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.787367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da22ea-c980-4a4f-8ed4-e40293d42951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.803589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-22da22ea-c980-4a4f-8ed4-e40293d42951 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:37.804064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 551/1087] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:41:37.804724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-22da22ea-c980-4a4f-8ed4-e40293d42951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:37.810224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db8eb1-8cc4-4bad-8303-e77809cacc56 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.872711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.881237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 537/1088] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:37.904433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da22ea-c980-4a4f-8ed4-e40293d42951 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.918436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db8eb1-8cc4-4bad-8303-e77809cacc56 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.952652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b8c9a-28dc-4975-9402-74fcd7d514d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.953439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b8c9a-28dc-4975-9402-74fcd7d514d3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.953714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b8c9a-28dc-4975-9402-74fcd7d514d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.953984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b8c9a-28dc-4975-9402-74fcd7d514d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.963786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b8c9a-28dc-4975-9402-74fcd7d514d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.965226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b8c9a-28dc-4975-9402-74fcd7d514d3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.973317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-e54b8c9a-28dc-4975-9402-74fcd7d514d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:37.982497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:37.988569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:37.989580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:37.989580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:37.989580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:37.989580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:37.989580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 552/1089] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:37.989580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b8c9a-28dc-4975-9402-74fcd7d514d3 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.001137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da22ea-c980-4a4f-8ed4-e40293d42951 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.035523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88beef4a-f485-431c-b40d-48c82d4b52ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.036337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88beef4a-f485-431c-b40d-48c82d4b52ec None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.036614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88beef4a-f485-431c-b40d-48c82d4b52ec None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.036892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88beef4a-f485-431c-b40d-48c82d4b52ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.047185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88beef4a-f485-431c-b40d-48c82d4b52ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-88beef4a-f485-431c-b40d-48c82d4b52ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:38.054817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:38.055969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:41:38.056831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 538/1090] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:38.061763 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88beef4a-f485-431c-b40d-48c82d4b52ec None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.086051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b8c9a-28dc-4975-9402-74fcd7d514d3 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.116772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.119649 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597f7a1-ba6a-4912-a7c6-65e1dd749687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.120311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597f7a1-ba6a-4912-a7c6-65e1dd749687 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.120591 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597f7a1-ba6a-4912-a7c6-65e1dd749687 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.120863 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597f7a1-ba6a-4912-a7c6-65e1dd749687 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.121585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597f7a1-ba6a-4912-a7c6-65e1dd749687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:38.121893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597f7a1-ba6a-4912-a7c6-65e1dd749687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.123301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597f7a1-ba6a-4912-a7c6-65e1dd749687 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.139652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 553/1091] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:38.140698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597f7a1-ba6a-4912-a7c6-65e1dd749687 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.159441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88beef4a-f485-431c-b40d-48c82d4b52ec None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.194350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4ae29-90ba-43e0-a4aa-21749927e46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.195060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4ae29-90ba-43e0-a4aa-21749927e46b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.195340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4ae29-90ba-43e0-a4aa-21749927e46b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.195611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4ae29-90ba-43e0-a4aa-21749927e46b None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.196886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4ae29-90ba-43e0-a4aa-21749927e46b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:38.197190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4ae29-90ba-43e0-a4aa-21749927e46b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.198581 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4ae29-90ba-43e0-a4aa-21749927e46b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.206724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 539/1092] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:38.211307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.213080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4ae29-90ba-43e0-a4aa-21749927e46b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.251398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597f7a1-ba6a-4912-a7c6-65e1dd749687 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.284901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613ace6-bf0a-4acc-8df9-6094662420c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.285690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613ace6-bf0a-4acc-8df9-6094662420c5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.285971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613ace6-bf0a-4acc-8df9-6094662420c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.286242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613ace6-bf0a-4acc-8df9-6094662420c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.292991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613ace6-bf0a-4acc-8df9-6094662420c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.294463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613ace6-bf0a-4acc-8df9-6094662420c5 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-e613ace6-bf0a-4acc-8df9-6094662420c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:38.308527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:38.309485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:38.310410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:38.310410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:38.310410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:38.310410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:38.310410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:38.310410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:38.310410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:38.310410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:38.310410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 554/1093] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:38.310952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4ae29-90ba-43e0-a4aa-21749927e46b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.315372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613ace6-bf0a-4acc-8df9-6094662420c5 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.327406 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.344516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16812fc-76cc-45cf-8a42-6286cfa860f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.345390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16812fc-76cc-45cf-8a42-6286cfa860f2 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.345667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16812fc-76cc-45cf-8a42-6286cfa860f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.345943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16812fc-76cc-45cf-8a42-6286cfa860f2 None admin] PATH_INFO: `/v3/users/ac66985266e8416b81dcfd913af3c165` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.347135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16812fc-76cc-45cf-8a42-6286cfa860f2 None admin] RBAC: Authorizing `identity:get_user(user_id=ac66985266e8416b81dcfd913af3c165)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.348558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16812fc-76cc-45cf-8a42-6286cfa860f2 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.350200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 540/1094] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/users/ac66985266e8416b81dcfd913af3c165 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:38.355824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16812fc-76cc-45cf-8a42-6286cfa860f2 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.413031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613ace6-bf0a-4acc-8df9-6094662420c5 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.433104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.447000 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81ddc6-c5b8-450a-976a-117f2cb70b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.447653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ddc6-c5b8-450a-976a-117f2cb70b3e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.447930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ddc6-c5b8-450a-976a-117f2cb70b3e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.448199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ddc6-c5b8-450a-976a-117f2cb70b3e None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.448976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81ddc6-c5b8-450a-976a-117f2cb70b3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:38.449280 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81ddc6-c5b8-450a-976a-117f2cb70b3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.450709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81ddc6-c5b8-450a-976a-117f2cb70b3e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.453659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16812fc-76cc-45cf-8a42-6286cfa860f2 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.461772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 555/1095] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:38.468433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81ddc6-c5b8-450a-976a-117f2cb70b3e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.488492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40541d9e-1a3f-40bc-aaf7-b5c8e193d65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.489846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40541d9e-1a3f-40bc-aaf7-b5c8e193d65c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.490216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40541d9e-1a3f-40bc-aaf7-b5c8e193d65c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.490591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40541d9e-1a3f-40bc-aaf7-b5c8e193d65c None admin] PATH_INFO: `/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.491889 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40541d9e-1a3f-40bc-aaf7-b5c8e193d65c None admin] RBAC: Authorizing `identity:get_project(project_id=1a0f2a9e182d4ef494ab1a2bae7c2c63)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.494235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40541d9e-1a3f-40bc-aaf7-b5c8e193d65c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.495985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 541/1096] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:38.501826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40541d9e-1a3f-40bc-aaf7-b5c8e193d65c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.559037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.572263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81ddc6-c5b8-450a-976a-117f2cb70b3e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.602614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40541d9e-1a3f-40bc-aaf7-b5c8e193d65c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.604234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272654a-1766-4b7c-8df0-c64f50ee1018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.604886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272654a-1766-4b7c-8df0-c64f50ee1018 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.605168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272654a-1766-4b7c-8df0-c64f50ee1018 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.605443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272654a-1766-4b7c-8df0-c64f50ee1018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.606266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272654a-1766-4b7c-8df0-c64f50ee1018 None admin] RBAC: Adding query filter params (role.id=333abb19b7ef46c7a17f4f86c8d798e3, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=067b827625e0427294b93b027597a5f3) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:38.606611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272654a-1766-4b7c-8df0-c64f50ee1018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.608041 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272654a-1766-4b7c-8df0-c64f50ee1018 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.613870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 556/1097] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/role_assignments?user.id=067b827625e0427294b93b027597a5f3&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=333abb19b7ef46c7a17f4f86c8d798e3 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:38.637449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f89ba-6dce-4dfe-885f-2951b45d8209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.638363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f89ba-6dce-4dfe-885f-2951b45d8209 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.638673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f89ba-6dce-4dfe-885f-2951b45d8209 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.639215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f89ba-6dce-4dfe-885f-2951b45d8209 None admin] PATH_INFO: `/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63/users/ac66985266e8416b81dcfd913af3c165/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.641291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f89ba-6dce-4dfe-885f-2951b45d8209 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0f2a9e182d4ef494ab1a2bae7c2c63, user_id=ac66985266e8416b81dcfd913af3c165, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.642789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f89ba-6dce-4dfe-885f-2951b45d8209 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.744985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 542/1098] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:38 2022] PUT /identity/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63/users/ac66985266e8416b81dcfd913af3c165/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:41:40.739740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d4235-2405-4e84-8550-c0f3a52af940 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.740269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d4235-2405-4e84-8550-c0f3a52af940 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.740269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d4235-2405-4e84-8550-c0f3a52af940 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.741671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 557/1099] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:40.746946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda926-bcb0-4ef6-9097-ac2b8beb4167 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.747370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda926-bcb0-4ef6-9097-ac2b8beb4167 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.747512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda926-bcb0-4ef6-9097-ac2b8beb4167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.764678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-14fda926-bcb0-4ef6-9097-ac2b8beb4167 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:40.765866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-14fda926-bcb0-4ef6-9097-ac2b8beb4167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:40.831102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 543/1100] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:40.854115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d4235-2405-4e84-8550-c0f3a52af940 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:40.901401 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:40.957233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d4235-2405-4e84-8550-c0f3a52af940 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:40.988650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd11bda-591c-4a89-9009-221a2540bbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:40.989428 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11bda-591c-4a89-9009-221a2540bbc9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.989705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11bda-591c-4a89-9009-221a2540bbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.990028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd11bda-591c-4a89-9009-221a2540bbc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.000305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd11bda-591c-4a89-9009-221a2540bbc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5bd11bda-591c-4a89-9009-221a2540bbc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:41.007795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:41.009452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:41:41.010616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 558/1101] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:41.015838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fda926-bcb0-4ef6-9097-ac2b8beb4167 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.059558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:41.111035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fda926-bcb0-4ef6-9097-ac2b8beb4167 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.146067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb8ae8-bd23-4c13-898d-0cb0f6d674fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.146751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb8ae8-bd23-4c13-898d-0cb0f6d674fe None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.147034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb8ae8-bd23-4c13-898d-0cb0f6d674fe None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.147307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb8ae8-bd23-4c13-898d-0cb0f6d674fe None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.148042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb8ae8-bd23-4c13-898d-0cb0f6d674fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:41.148348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb8ae8-bd23-4c13-898d-0cb0f6d674fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.149732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb8ae8-bd23-4c13-898d-0cb0f6d674fe None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.157540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 544/1102] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:41.163272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd11bda-591c-4a89-9009-221a2540bbc9 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.259266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd11bda-591c-4a89-9009-221a2540bbc9 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.261439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:41.292365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b1372-63dc-4cc4-a30e-fde9acb4513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.293217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b1372-63dc-4cc4-a30e-fde9acb4513e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.293502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b1372-63dc-4cc4-a30e-fde9acb4513e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.293780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b1372-63dc-4cc4-a30e-fde9acb4513e None admin] PATH_INFO: `/v3/users/ac66985266e8416b81dcfd913af3c165` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.294981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b1372-63dc-4cc4-a30e-fde9acb4513e None admin] RBAC: Authorizing `identity:get_user(user_id=ac66985266e8416b81dcfd913af3c165)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.296419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b1372-63dc-4cc4-a30e-fde9acb4513e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.298036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 559/1103] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/users/ac66985266e8416b81dcfd913af3c165 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:41.303564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb8ae8-bd23-4c13-898d-0cb0f6d674fe None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.398906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb8ae8-bd23-4c13-898d-0cb0f6d674fe None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.406035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:41.431297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52c6c5-00e5-4652-a1c7-5cba34bfa61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.431951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52c6c5-00e5-4652-a1c7-5cba34bfa61d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.432213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52c6c5-00e5-4652-a1c7-5cba34bfa61d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.432491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52c6c5-00e5-4652-a1c7-5cba34bfa61d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.496463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52c6c5-00e5-4652-a1c7-5cba34bfa61d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.497894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52c6c5-00e5-4652-a1c7-5cba34bfa61d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.592371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 545/1104] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:41.598253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b1372-63dc-4cc4-a30e-fde9acb4513e None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.694419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b1372-63dc-4cc4-a30e-fde9acb4513e None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.727237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71157005-f299-43d8-9632-9b3fedcabd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.728671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71157005-f299-43d8-9632-9b3fedcabd99 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.728951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71157005-f299-43d8-9632-9b3fedcabd99 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.729221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71157005-f299-43d8-9632-9b3fedcabd99 None admin] PATH_INFO: `/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.730399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71157005-f299-43d8-9632-9b3fedcabd99 None admin] RBAC: Authorizing `identity:get_project(project_id=1a0f2a9e182d4ef494ab1a2bae7c2c63)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.731852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71157005-f299-43d8-9632-9b3fedcabd99 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.733515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 560/1105] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:41.739570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52c6c5-00e5-4652-a1c7-5cba34bfa61d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.747367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:41.835540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52c6c5-00e5-4652-a1c7-5cba34bfa61d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.868757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde29ea-7a2d-4f2f-a16f-23a76ca124f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.869344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde29ea-7a2d-4f2f-a16f-23a76ca124f8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.869626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde29ea-7a2d-4f2f-a16f-23a76ca124f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.869899 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde29ea-7a2d-4f2f-a16f-23a76ca124f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.870742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde29ea-7a2d-4f2f-a16f-23a76ca124f8 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=1a0f2a9e182d4ef494ab1a2bae7c2c63, user.id=ac66985266e8416b81dcfd913af3c165) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:41.871053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde29ea-7a2d-4f2f-a16f-23a76ca124f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.872462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde29ea-7a2d-4f2f-a16f-23a76ca124f8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.878139 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 546/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/role_assignments?user.id=ac66985266e8416b81dcfd913af3c165&scope.project.id=1a0f2a9e182d4ef494ab1a2bae7c2c63&role.id=7a833c45603a4830b888e58e1232712a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:43.887258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9ee8-69e7-4c61-ae77-59f511f4f8cb None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.887823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9ee8-69e7-4c61-ae77-59f511f4f8cb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.887823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f9ee8-69e7-4c61-ae77-59f511f4f8cb None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.889457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 561/1107] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:43.894743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf3d96-b8bc-4aa0-b5d1-ea561ed040dd None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.895167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf3d96-b8bc-4aa0-b5d1-ea561ed040dd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.895294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf3d96-b8bc-4aa0-b5d1-ea561ed040dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.911611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-dcbf3d96-b8bc-4aa0-b5d1-ea561ed040dd None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:43.912791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-dcbf3d96-b8bc-4aa0-b5d1-ea561ed040dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:43.973219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:43.977690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 547/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:44.002653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9ee8-69e7-4c61-ae77-59f511f4f8cb None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:44.044561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:44.100545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f9ee8-69e7-4c61-ae77-59f511f4f8cb None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:44.134407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0c354-1d40-4be2-b2d0-a84a0f20c985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:44.135222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0c354-1d40-4be2-b2d0-a84a0f20c985 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:44.135507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0c354-1d40-4be2-b2d0-a84a0f20c985 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:44.135781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0c354-1d40-4be2-b2d0-a84a0f20c985 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:44.136950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0c354-1d40-4be2-b2d0-a84a0f20c985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:44.138436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0c354-1d40-4be2-b2d0-a84a0f20c985 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:44.140290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 562/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:44.145965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf3d96-b8bc-4aa0-b5d1-ea561ed040dd None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:44.240329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf3d96-b8bc-4aa0-b5d1-ea561ed040dd None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:44.272761 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492aad0-b7cd-4c38-9bc7-29c1838e321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:44.273408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492aad0-b7cd-4c38-9bc7-29c1838e321e None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:44.273685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492aad0-b7cd-4c38-9bc7-29c1838e321e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:44.273957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492aad0-b7cd-4c38-9bc7-29c1838e321e None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:44.274717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492aad0-b7cd-4c38-9bc7-29c1838e321e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:44.276131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492aad0-b7cd-4c38-9bc7-29c1838e321e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:44.372365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 548/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:41:44 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:46.415611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cf5c8-2d80-4a90-88cf-7efa1ea5bab5 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.416200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cf5c8-2d80-4a90-88cf-7efa1ea5bab5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.416200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cf5c8-2d80-4a90-88cf-7efa1ea5bab5 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.417550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 563/1111] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:46.422610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba1001-0d35-4ad9-b3aa-530fe78c58f9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.423073 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba1001-0d35-4ad9-b3aa-530fe78c58f9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.423177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba1001-0d35-4ad9-b3aa-530fe78c58f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.439972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-95ba1001-0d35-4ad9-b3aa-530fe78c58f9 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:46.441138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-95ba1001-0d35-4ad9-b3aa-530fe78c58f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:46.488578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:46.504668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 549/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:46.527806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cf5c8-2d80-4a90-88cf-7efa1ea5bab5 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:46.626642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cf5c8-2d80-4a90-88cf-7efa1ea5bab5 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:46.636077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:46.660489 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd840f-ec46-4668-8fdf-59e583afcecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:46.661342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd840f-ec46-4668-8fdf-59e583afcecf None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.661628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd840f-ec46-4668-8fdf-59e583afcecf None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.661902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd840f-ec46-4668-8fdf-59e583afcecf None admin] PATH_INFO: `/v3/roles/73a019c7fb8340a58c7dfe4550341949` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.663404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd840f-ec46-4668-8fdf-59e583afcecf None admin] RBAC: Authorizing `identity:get_role(role_id=73a019c7fb8340a58c7dfe4550341949)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:46.664805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd840f-ec46-4668-8fdf-59e583afcecf None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:46.666230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 564/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:41:46 2022] GET /identity/v3/roles/73a019c7fb8340a58c7dfe4550341949 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:46.671928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba1001-0d35-4ad9-b3aa-530fe78c58f9 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:46.765635 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba1001-0d35-4ad9-b3aa-530fe78c58f9 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:46.797670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8cd1b-d9eb-4aa9-92c0-55af67043afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:46.798543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8cd1b-d9eb-4aa9-92c0-55af67043afd None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.798821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8cd1b-d9eb-4aa9-92c0-55af67043afd None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.799091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8cd1b-d9eb-4aa9-92c0-55af67043afd None admin] PATH_INFO: `/v3/users/63989d5264c34ab89b3c5c8209fbc291` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.812038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8cd1b-d9eb-4aa9-92c0-55af67043afd None admin] RBAC: Authorizing `identity:get_user(user_id=63989d5264c34ab89b3c5c8209fbc291)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:46.813458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8cd1b-d9eb-4aa9-92c0-55af67043afd None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:46.815091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 550/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:41:46 2022] GET /identity/v3/users/63989d5264c34ab89b3c5c8209fbc291 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:46.820577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd840f-ec46-4668-8fdf-59e583afcecf None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:46.915875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd840f-ec46-4668-8fdf-59e583afcecf None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:46.949517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afc291-a8bf-4307-9002-0730e86dbc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:46.950092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afc291-a8bf-4307-9002-0730e86dbc77 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.950376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afc291-a8bf-4307-9002-0730e86dbc77 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.950686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afc291-a8bf-4307-9002-0730e86dbc77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.992462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:47.015038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afc291-a8bf-4307-9002-0730e86dbc77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.016479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afc291-a8bf-4307-9002-0730e86dbc77 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.109328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 565/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:47.115350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8cd1b-d9eb-4aa9-92c0-55af67043afd None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.160510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:47.213134 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8cd1b-d9eb-4aa9-92c0-55af67043afd None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.246493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16f3da-6749-4063-a7d6-867bf835f2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.247355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16f3da-6749-4063-a7d6-867bf835f2eb None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.247634 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16f3da-6749-4063-a7d6-867bf835f2eb None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.247906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16f3da-6749-4063-a7d6-867bf835f2eb None admin] PATH_INFO: `/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.249059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16f3da-6749-4063-a7d6-867bf835f2eb None admin] RBAC: Authorizing `identity:get_project(project_id=1a0f2a9e182d4ef494ab1a2bae7c2c63)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.250499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16f3da-6749-4063-a7d6-867bf835f2eb None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.252158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 551/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:47.258165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afc291-a8bf-4307-9002-0730e86dbc77 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.330723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:47.351865 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afc291-a8bf-4307-9002-0730e86dbc77 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.386029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55553170-3366-4ad6-b74a-bf2bce06711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.386721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55553170-3366-4ad6-b74a-bf2bce06711b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.387004 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55553170-3366-4ad6-b74a-bf2bce06711b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.387276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55553170-3366-4ad6-b74a-bf2bce06711b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.388095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55553170-3366-4ad6-b74a-bf2bce06711b None admin] RBAC: Adding query filter params (role.id=73a019c7fb8340a58c7dfe4550341949, scope.project.id=1a0f2a9e182d4ef494ab1a2bae7c2c63, user.id=63989d5264c34ab89b3c5c8209fbc291) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:47.388402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55553170-3366-4ad6-b74a-bf2bce06711b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.390080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55553170-3366-4ad6-b74a-bf2bce06711b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.395367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 566/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/role_assignments?user.id=63989d5264c34ab89b3c5c8209fbc291&scope.project.id=1a0f2a9e182d4ef494ab1a2bae7c2c63&role.id=73a019c7fb8340a58c7dfe4550341949 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:49.378338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0aa38-677d-4e3b-9022-b1454d6543e5 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.378916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0aa38-677d-4e3b-9022-b1454d6543e5 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.378916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0aa38-677d-4e3b-9022-b1454d6543e5 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.380277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 552/1118] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:49.385941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa599271-cac6-4f7b-bbcc-84278f202caa None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.386357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa599271-cac6-4f7b-bbcc-84278f202caa None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.386511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa599271-cac6-4f7b-bbcc-84278f202caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.403589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-fa599271-cac6-4f7b-bbcc-84278f202caa None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:49.404800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-fa599271-cac6-4f7b-bbcc-84278f202caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:49.467938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 567/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:49.489922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0aa38-677d-4e3b-9022-b1454d6543e5 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.588377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0aa38-677d-4e3b-9022-b1454d6543e5 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.600893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:49.620534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dad69f-666a-40e1-a7ce-e7d77b8a868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.621386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dad69f-666a-40e1-a7ce-e7d77b8a868e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.621664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dad69f-666a-40e1-a7ce-e7d77b8a868e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.621936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dad69f-666a-40e1-a7ce-e7d77b8a868e None admin] PATH_INFO: `/v3/roles/73a019c7fb8340a58c7dfe4550341949` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.623401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dad69f-666a-40e1-a7ce-e7d77b8a868e None admin] RBAC: Authorizing `identity:get_role(role_id=73a019c7fb8340a58c7dfe4550341949)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.624755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dad69f-666a-40e1-a7ce-e7d77b8a868e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.626113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 553/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:41:49 2022] GET /identity/v3/roles/73a019c7fb8340a58c7dfe4550341949 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:49.632016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa599271-cac6-4f7b-bbcc-84278f202caa None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.698849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:49.727795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa599271-cac6-4f7b-bbcc-84278f202caa None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.760293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9358f-4f12-4e0d-9475-6f4d561fd0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.761145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9358f-4f12-4e0d-9475-6f4d561fd0f6 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.761421 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9358f-4f12-4e0d-9475-6f4d561fd0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.761692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9358f-4f12-4e0d-9475-6f4d561fd0f6 None admin] PATH_INFO: `/v3/users/63989d5264c34ab89b3c5c8209fbc291` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.762875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9358f-4f12-4e0d-9475-6f4d561fd0f6 None admin] RBAC: Authorizing `identity:get_user(user_id=63989d5264c34ab89b3c5c8209fbc291)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.764566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9358f-4f12-4e0d-9475-6f4d561fd0f6 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.766166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 568/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:41:49 2022] GET /identity/v3/users/63989d5264c34ab89b3c5c8209fbc291 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:49.772098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dad69f-666a-40e1-a7ce-e7d77b8a868e None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.867306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dad69f-666a-40e1-a7ce-e7d77b8a868e None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.901677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae06e62-7cf3-4bf7-be24-67b24bf9b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.902455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae06e62-7cf3-4bf7-be24-67b24bf9b42a None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.902755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae06e62-7cf3-4bf7-be24-67b24bf9b42a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.903034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae06e62-7cf3-4bf7-be24-67b24bf9b42a None admin] PATH_INFO: `/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.904200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae06e62-7cf3-4bf7-be24-67b24bf9b42a None admin] RBAC: Authorizing `identity:get_project(project_id=1a0f2a9e182d4ef494ab1a2bae7c2c63)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.905629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae06e62-7cf3-4bf7-be24-67b24bf9b42a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.907804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 554/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:49 2022] GET /identity/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:49.913509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9358f-4f12-4e0d-9475-6f4d561fd0f6 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:50.008334 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9358f-4f12-4e0d-9475-6f4d561fd0f6 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:50.040852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b05f67-1ab7-4913-b0ae-b611ac6c8bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:50.041714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b05f67-1ab7-4913-b0ae-b611ac6c8bc7 None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:50.041997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b05f67-1ab7-4913-b0ae-b611ac6c8bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:50.042275 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b05f67-1ab7-4913-b0ae-b611ac6c8bc7 None admin] PATH_INFO: `/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63/users/63989d5264c34ab89b3c5c8209fbc291/roles/73a019c7fb8340a58c7dfe4550341949` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:50.044475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b05f67-1ab7-4913-b0ae-b611ac6c8bc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0f2a9e182d4ef494ab1a2bae7c2c63, user_id=63989d5264c34ab89b3c5c8209fbc291, role_id=73a019c7fb8340a58c7dfe4550341949)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:50.045898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b05f67-1ab7-4913-b0ae-b611ac6c8bc7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:50.099128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:50.104311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 569/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:49 2022] PUT /identity/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63/users/63989d5264c34ab89b3c5c8209fbc291/roles/73a019c7fb8340a58c7dfe4550341949 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:41:52.068960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4cd3b-2921-4632-8202-b668249a34d9 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.069474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4cd3b-2921-4632-8202-b668249a34d9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.069474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4cd3b-2921-4632-8202-b668249a34d9 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.071437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 555/1124] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:52.076480 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39624c99-3e2c-4288-8f80-b1ef1b7b3010 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.076889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39624c99-3e2c-4288-8f80-b1ef1b7b3010 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.077025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39624c99-3e2c-4288-8f80-b1ef1b7b3010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.093143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-39624c99-3e2c-4288-8f80-b1ef1b7b3010 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:52.094311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-39624c99-3e2c-4288-8f80-b1ef1b7b3010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:52.157626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 570/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:52.179526 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4cd3b-2921-4632-8202-b668249a34d9 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.230463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:52.279912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4cd3b-2921-4632-8202-b668249a34d9 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.312701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ab41e-fded-4254-bd65-bde89e04b1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.313546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ab41e-fded-4254-bd65-bde89e04b1a9 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.313823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ab41e-fded-4254-bd65-bde89e04b1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.314096 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ab41e-fded-4254-bd65-bde89e04b1a9 None admin] PATH_INFO: `/v3/roles/73a019c7fb8340a58c7dfe4550341949` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.315562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ab41e-fded-4254-bd65-bde89e04b1a9 None admin] RBAC: Authorizing `identity:get_role(role_id=73a019c7fb8340a58c7dfe4550341949)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.316914 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ab41e-fded-4254-bd65-bde89e04b1a9 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.318350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 556/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/roles/73a019c7fb8340a58c7dfe4550341949 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:52.324926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39624c99-3e2c-4288-8f80-b1ef1b7b3010 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.424606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39624c99-3e2c-4288-8f80-b1ef1b7b3010 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.450005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:52.457472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ca757-6f18-42ec-9ef0-e1a7493073c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.461162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ca757-6f18-42ec-9ef0-e1a7493073c5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.461162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ca757-6f18-42ec-9ef0-e1a7493073c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.461162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ca757-6f18-42ec-9ef0-e1a7493073c5 None admin] PATH_INFO: `/v3/users/63989d5264c34ab89b3c5c8209fbc291` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.461162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ca757-6f18-42ec-9ef0-e1a7493073c5 None admin] RBAC: Authorizing `identity:get_user(user_id=63989d5264c34ab89b3c5c8209fbc291)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.461517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ca757-6f18-42ec-9ef0-e1a7493073c5 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.463157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 571/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/users/63989d5264c34ab89b3c5c8209fbc291 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:52.468807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ab41e-fded-4254-bd65-bde89e04b1a9 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.564020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ab41e-fded-4254-bd65-bde89e04b1a9 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.583935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:52.604457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040686b4-8eb3-4ac6-a44b-61b06c1b5c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.605118 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040686b4-8eb3-4ac6-a44b-61b06c1b5c7b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.605525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040686b4-8eb3-4ac6-a44b-61b06c1b5c7b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.605809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040686b4-8eb3-4ac6-a44b-61b06c1b5c7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.671517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040686b4-8eb3-4ac6-a44b-61b06c1b5c7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.672958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040686b4-8eb3-4ac6-a44b-61b06c1b5c7b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.767558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 557/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:52.773362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ca757-6f18-42ec-9ef0-e1a7493073c5 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.869143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ca757-6f18-42ec-9ef0-e1a7493073c5 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.903202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0676e-2816-48d4-a855-3a7655bde2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.904055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0676e-2816-48d4-a855-3a7655bde2bd None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.904338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0676e-2816-48d4-a855-3a7655bde2bd None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.904613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0676e-2816-48d4-a855-3a7655bde2bd None admin] PATH_INFO: `/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.905797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0676e-2816-48d4-a855-3a7655bde2bd None admin] RBAC: Authorizing `identity:get_project(project_id=1a0f2a9e182d4ef494ab1a2bae7c2c63)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.907716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0676e-2816-48d4-a855-3a7655bde2bd None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.908981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 572/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/projects/1a0f2a9e182d4ef494ab1a2bae7c2c63 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:52.915720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040686b4-8eb3-4ac6-a44b-61b06c1b5c7b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.934516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:53.021480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040686b4-8eb3-4ac6-a44b-61b06c1b5c7b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:53.054892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70727496-bb87-4717-866a-23bdfdf0d01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:53.055659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70727496-bb87-4717-866a-23bdfdf0d01e None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.055941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70727496-bb87-4717-866a-23bdfdf0d01e None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.056216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70727496-bb87-4717-866a-23bdfdf0d01e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.057036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70727496-bb87-4717-866a-23bdfdf0d01e None admin] RBAC: Adding query filter params (role.id=73a019c7fb8340a58c7dfe4550341949, scope.project.id=1a0f2a9e182d4ef494ab1a2bae7c2c63, user.id=63989d5264c34ab89b3c5c8209fbc291) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:53.057347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70727496-bb87-4717-866a-23bdfdf0d01e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:53.058784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70727496-bb87-4717-866a-23bdfdf0d01e None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:53.064456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 558/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/role_assignments?user.id=63989d5264c34ab89b3c5c8209fbc291&scope.project.id=1a0f2a9e182d4ef494ab1a2bae7c2c63&role.id=73a019c7fb8340a58c7dfe4550341949 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:55.090572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940632-0da9-4869-8fb7-fed7c43200e1 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.091119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940632-0da9-4869-8fb7-fed7c43200e1 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.091119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940632-0da9-4869-8fb7-fed7c43200e1 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.092510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 573/1131] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:55.097624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62354de3-5117-4ed9-9b8b-ffde55535aba None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.098057 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62354de3-5117-4ed9-9b8b-ffde55535aba None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.098161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62354de3-5117-4ed9-9b8b-ffde55535aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.114290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-62354de3-5117-4ed9-9b8b-ffde55535aba None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:55.115487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-62354de3-5117-4ed9-9b8b-ffde55535aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:55.178871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 559/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:55.202346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb940632-0da9-4869-8fb7-fed7c43200e1 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:55.263770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:55.300686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb940632-0da9-4869-8fb7-fed7c43200e1 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.333105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24348681-6235-4ebb-a041-7ac3d1de42a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:55.333882 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24348681-6235-4ebb-a041-7ac3d1de42a2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.334165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24348681-6235-4ebb-a041-7ac3d1de42a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.334441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24348681-6235-4ebb-a041-7ac3d1de42a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.335628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24348681-6235-4ebb-a041-7ac3d1de42a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:55.337095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24348681-6235-4ebb-a041-7ac3d1de42a2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:55.338750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 574/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:55.344393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62354de3-5117-4ed9-9b8b-ffde55535aba None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:55.355739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:55.440649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62354de3-5117-4ed9-9b8b-ffde55535aba None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.472494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea055a-257c-4221-9f6c-d876522ef190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:55.473209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea055a-257c-4221-9f6c-d876522ef190 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.473455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea055a-257c-4221-9f6c-d876522ef190 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.473728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea055a-257c-4221-9f6c-d876522ef190 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.474434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea055a-257c-4221-9f6c-d876522ef190 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:55.475879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea055a-257c-4221-9f6c-d876522ef190 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:55.583848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 560/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:41:55 2022] POST /identity/v3/projects => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:57.622616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dbee5-a2c4-4a70-8055-e30c1a474a88 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:57.623174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dbee5-a2c4-4a70-8055-e30c1a474a88 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:57.623174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dbee5-a2c4-4a70-8055-e30c1a474a88 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:57.624569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 575/1135] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:57.629917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecccd5a-ad4f-4c29-aeb8-a607442975b8 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:57.630316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecccd5a-ad4f-4c29-aeb8-a607442975b8 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:57.630492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecccd5a-ad4f-4c29-aeb8-a607442975b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:57.647522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3ecccd5a-ad4f-4c29-aeb8-a607442975b8 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:57.648700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3ecccd5a-ad4f-4c29-aeb8-a607442975b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:57.703416 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:57.713857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 561/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:57.738195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dbee5-a2c4-4a70-8055-e30c1a474a88 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:57.837172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dbee5-a2c4-4a70-8055-e30c1a474a88 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:57.864791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:57.869972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25d2d7-ac28-4205-9fac-25561769efe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:57.870781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25d2d7-ac28-4205-9fac-25561769efe9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:57.871060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25d2d7-ac28-4205-9fac-25561769efe9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:57.871333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25d2d7-ac28-4205-9fac-25561769efe9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:57.872504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25d2d7-ac28-4205-9fac-25561769efe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:57.876570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25d2d7-ac28-4205-9fac-25561769efe9 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:57.876570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 576/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:57.881535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecccd5a-ad4f-4c29-aeb8-a607442975b8 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:57.979845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecccd5a-ad4f-4c29-aeb8-a607442975b8 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:58.012267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d2248-6a09-4a4e-8417-11190aebf2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:58.012921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2248-6a09-4a4e-8417-11190aebf2d0 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.013199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2248-6a09-4a4e-8417-11190aebf2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.013472 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2248-6a09-4a4e-8417-11190aebf2d0 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.014429 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d2248-6a09-4a4e-8417-11190aebf2d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:58.016016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d2248-6a09-4a4e-8417-11190aebf2d0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:58.114075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 562/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:41:57 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:00.165017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931c22-86b6-490d-945e-de615454ac8d None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.165515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931c22-86b6-490d-945e-de615454ac8d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.165515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931c22-86b6-490d-945e-de615454ac8d None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.166905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 577/1139] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:00.172066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334e479-07a2-4732-8269-c3589c41f4d7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.172502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334e479-07a2-4732-8269-c3589c41f4d7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.172626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334e479-07a2-4732-8269-c3589c41f4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.188892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-9334e479-07a2-4732-8269-c3589c41f4d7 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:00.190071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9334e479-07a2-4732-8269-c3589c41f4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:00.232345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:00.253965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 563/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:00.277541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5931c22-86b6-490d-945e-de615454ac8d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:00.374175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5931c22-86b6-490d-945e-de615454ac8d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:00.406325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fe46d-1c56-4f36-a723-5d3186f085c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:00.407127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fe46d-1c56-4f36-a723-5d3186f085c0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.407404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fe46d-1c56-4f36-a723-5d3186f085c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.407674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fe46d-1c56-4f36-a723-5d3186f085c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.417489 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19fe46d-1c56-4f36-a723-5d3186f085c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-a19fe46d-1c56-4f36-a723-5d3186f085c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:00.425062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:00.426235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:42:00.427536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 578/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:00.432592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334e479-07a2-4732-8269-c3589c41f4d7 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:00.528011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334e479-07a2-4732-8269-c3589c41f4d7 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:00.560478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975201b-9437-4619-8b15-6cc6efb06b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:00.561130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975201b-9437-4619-8b15-6cc6efb06b41 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.561411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975201b-9437-4619-8b15-6cc6efb06b41 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.561683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975201b-9437-4619-8b15-6cc6efb06b41 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.562415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975201b-9437-4619-8b15-6cc6efb06b41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:00.562749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975201b-9437-4619-8b15-6cc6efb06b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:00.564102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975201b-9437-4619-8b15-6cc6efb06b41 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:00.572438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 564/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:00.578014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fe46d-1c56-4f36-a723-5d3186f085c0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:00.635382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:00.677071 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fe46d-1c56-4f36-a723-5d3186f085c0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:00.709443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3959ea-7d45-4b60-baa1-07457587f732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:00.710302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3959ea-7d45-4b60-baa1-07457587f732 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.710623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3959ea-7d45-4b60-baa1-07457587f732 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.710903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3959ea-7d45-4b60-baa1-07457587f732 None admin] PATH_INFO: `/v3/users/ba5b0caa6d8c4236912bea7c7b2483e5` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.723419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3959ea-7d45-4b60-baa1-07457587f732 None admin] RBAC: Authorizing `identity:get_user(user_id=ba5b0caa6d8c4236912bea7c7b2483e5)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:00.724866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3959ea-7d45-4b60-baa1-07457587f732 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:00.726524 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 579/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:00 2022] GET /identity/v3/users/ba5b0caa6d8c4236912bea7c7b2483e5 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:00.732642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975201b-9437-4619-8b15-6cc6efb06b41 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:00.773235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:00.830341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975201b-9437-4619-8b15-6cc6efb06b41 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:00.862558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269a95c-9d62-4d3d-8dab-46aff08bddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:00.863200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269a95c-9d62-4d3d-8dab-46aff08bddab None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.863479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269a95c-9d62-4d3d-8dab-46aff08bddab None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.863749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269a95c-9d62-4d3d-8dab-46aff08bddab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.928552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269a95c-9d62-4d3d-8dab-46aff08bddab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:00.930549 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269a95c-9d62-4d3d-8dab-46aff08bddab None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:00.976222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:01.042896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 565/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:01.048700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3959ea-7d45-4b60-baa1-07457587f732 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:01.143981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3959ea-7d45-4b60-baa1-07457587f732 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:01.160835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:01.177351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d08e4-d654-4551-bca1-0ee4310d23ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:01.178198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d08e4-d654-4551-bca1-0ee4310d23ea None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.178500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d08e4-d654-4551-bca1-0ee4310d23ea None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.178831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d08e4-d654-4551-bca1-0ee4310d23ea None admin] PATH_INFO: `/v3/projects/989513614f744c60ad2bc89a348c6871` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.180024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d08e4-d654-4551-bca1-0ee4310d23ea None admin] RBAC: Authorizing `identity:get_project(project_id=989513614f744c60ad2bc89a348c6871)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:01.181466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d08e4-d654-4551-bca1-0ee4310d23ea None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:01.183286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 580/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:01 2022] GET /identity/v3/projects/989513614f744c60ad2bc89a348c6871 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:01.189141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269a95c-9d62-4d3d-8dab-46aff08bddab None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:01.282819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269a95c-9d62-4d3d-8dab-46aff08bddab None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:01.319451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e0fde-dabe-4a59-913d-17452c79d8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:01.320095 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0fde-dabe-4a59-913d-17452c79d8d2 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.320373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0fde-dabe-4a59-913d-17452c79d8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.320669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0fde-dabe-4a59-913d-17452c79d8d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.321480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e0fde-dabe-4a59-913d-17452c79d8d2 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=989513614f744c60ad2bc89a348c6871, user.id=ba5b0caa6d8c4236912bea7c7b2483e5) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:01.321789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e0fde-dabe-4a59-913d-17452c79d8d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:01.323215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e0fde-dabe-4a59-913d-17452c79d8d2 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:01.328495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 566/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:01 2022] GET /identity/v3/role_assignments?user.id=ba5b0caa6d8c4236912bea7c7b2483e5&scope.project.id=989513614f744c60ad2bc89a348c6871&role.id=7a833c45603a4830b888e58e1232712a => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:03.292386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d17f7f-6233-4e66-84a2-ef5ac8e80349 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:03.292906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d17f7f-6233-4e66-84a2-ef5ac8e80349 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:03.292906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d17f7f-6233-4e66-84a2-ef5ac8e80349 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:03.294232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 581/1147] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:03.299474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d41b7-6bc3-478e-be1b-d4c76b9c3eed None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:03.299882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d41b7-6bc3-478e-be1b-d4c76b9c3eed None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:03.300032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d41b7-6bc3-478e-be1b-d4c76b9c3eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:03.306453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:03.318100 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-536d41b7-6bc3-478e-be1b-d4c76b9c3eed None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:03.319317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-536d41b7-6bc3-478e-be1b-d4c76b9c3eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:03.383947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 567/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:03.406265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d17f7f-6233-4e66-84a2-ef5ac8e80349 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:03.503734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d17f7f-6233-4e66-84a2-ef5ac8e80349 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:03.536866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b573e8-a37d-4159-bbf6-4a7da2b8ff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:03.537646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b573e8-a37d-4159-bbf6-4a7da2b8ff29 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:03.537925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b573e8-a37d-4159-bbf6-4a7da2b8ff29 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:03.538197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b573e8-a37d-4159-bbf6-4a7da2b8ff29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:03.547914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b573e8-a37d-4159-bbf6-4a7da2b8ff29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-d7b573e8-a37d-4159-bbf6-4a7da2b8ff29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:03.555875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:03.556983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:42:03.558056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 582/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:03.562850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d41b7-6bc3-478e-be1b-d4c76b9c3eed None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:03.657052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d41b7-6bc3-478e-be1b-d4c76b9c3eed None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:03.690291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee2681-b201-41d5-a5ca-b3cf2493977c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:03.690973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee2681-b201-41d5-a5ca-b3cf2493977c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:03.691251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee2681-b201-41d5-a5ca-b3cf2493977c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:03.691522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee2681-b201-41d5-a5ca-b3cf2493977c None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:03.692251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee2681-b201-41d5-a5ca-b3cf2493977c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:03.692552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee2681-b201-41d5-a5ca-b3cf2493977c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:03.693906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee2681-b201-41d5-a5ca-b3cf2493977c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:03.695237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:03.702060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 568/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:03.708005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b573e8-a37d-4159-bbf6-4a7da2b8ff29 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:03.753802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:03.809662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b573e8-a37d-4159-bbf6-4a7da2b8ff29 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:03.840623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f40de-f8e8-4653-a76d-de636c125ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:03.841479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f40de-f8e8-4653-a76d-de636c125ffe None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:03.841755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f40de-f8e8-4653-a76d-de636c125ffe None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:03.842024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f40de-f8e8-4653-a76d-de636c125ffe None admin] PATH_INFO: `/v3/users/ba5b0caa6d8c4236912bea7c7b2483e5` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:03.843206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f40de-f8e8-4653-a76d-de636c125ffe None admin] RBAC: Authorizing `identity:get_user(user_id=ba5b0caa6d8c4236912bea7c7b2483e5)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:03.844623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f40de-f8e8-4653-a76d-de636c125ffe None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:03.846222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 583/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:03 2022] GET /identity/v3/users/ba5b0caa6d8c4236912bea7c7b2483e5 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:03.852490 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee2681-b201-41d5-a5ca-b3cf2493977c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:03.973060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee2681-b201-41d5-a5ca-b3cf2493977c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:04.009531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103d455-2b76-42c9-a1d3-906304addbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:04.010384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103d455-2b76-42c9-a1d3-906304addbf1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.010689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103d455-2b76-42c9-a1d3-906304addbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.010964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103d455-2b76-42c9-a1d3-906304addbf1 None admin] PATH_INFO: `/v3/projects/989513614f744c60ad2bc89a348c6871` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.012132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103d455-2b76-42c9-a1d3-906304addbf1 None admin] RBAC: Authorizing `identity:get_project(project_id=989513614f744c60ad2bc89a348c6871)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:04.013569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103d455-2b76-42c9-a1d3-906304addbf1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:04.015566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 569/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:03 2022] GET /identity/v3/projects/989513614f744c60ad2bc89a348c6871 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:04.021450 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f40de-f8e8-4653-a76d-de636c125ffe None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:04.117990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f40de-f8e8-4653-a76d-de636c125ffe None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:04.131003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:04.150739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d87bc3-3522-4438-a0f8-7fd011eb52c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:04.151620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d87bc3-3522-4438-a0f8-7fd011eb52c7 None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.151901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d87bc3-3522-4438-a0f8-7fd011eb52c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.152177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d87bc3-3522-4438-a0f8-7fd011eb52c7 None admin] PATH_INFO: `/v3/projects/989513614f744c60ad2bc89a348c6871/users/ba5b0caa6d8c4236912bea7c7b2483e5/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.154122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d87bc3-3522-4438-a0f8-7fd011eb52c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=989513614f744c60ad2bc89a348c6871, user_id=ba5b0caa6d8c4236912bea7c7b2483e5, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:04.155583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d87bc3-3522-4438-a0f8-7fd011eb52c7 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:04.223292 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 584/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:42:04 2022] PUT /identity/v3/projects/989513614f744c60ad2bc89a348c6871/users/ba5b0caa6d8c4236912bea7c7b2483e5/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:06.336804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b11ed0-acbf-4caa-976d-4dcf3a8afa58 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.337301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b11ed0-acbf-4caa-976d-4dcf3a8afa58 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.337301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b11ed0-acbf-4caa-976d-4dcf3a8afa58 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.338705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 570/1154] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:06.343917 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cf577-e6ba-4496-bfa1-9198d7616c1a None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.344315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cf577-e6ba-4496-bfa1-9198d7616c1a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.344479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cf577-e6ba-4496-bfa1-9198d7616c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.360749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e55cf577-e6ba-4496-bfa1-9198d7616c1a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:06.361914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e55cf577-e6ba-4496-bfa1-9198d7616c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:06.426054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 585/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:06.449439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b11ed0-acbf-4caa-976d-4dcf3a8afa58 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:06.511740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:06.549518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b11ed0-acbf-4caa-976d-4dcf3a8afa58 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.582511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34995b8-9210-44d6-8cbc-beb2049ffdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.583290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34995b8-9210-44d6-8cbc-beb2049ffdc9 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.583572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34995b8-9210-44d6-8cbc-beb2049ffdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.583847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34995b8-9210-44d6-8cbc-beb2049ffdc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.593678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34995b8-9210-44d6-8cbc-beb2049ffdc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-b34995b8-9210-44d6-8cbc-beb2049ffdc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:06.601307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:06.602418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:42:06.603368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 571/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:06.608443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cf577-e6ba-4496-bfa1-9198d7616c1a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:06.708697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cf577-e6ba-4496-bfa1-9198d7616c1a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.711027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:06.742047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70dd1e-b51b-4593-86bd-745985e440c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.743437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70dd1e-b51b-4593-86bd-745985e440c2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.743722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70dd1e-b51b-4593-86bd-745985e440c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.743993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70dd1e-b51b-4593-86bd-745985e440c2 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.744775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70dd1e-b51b-4593-86bd-745985e440c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:06.745078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70dd1e-b51b-4593-86bd-745985e440c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:06.746444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70dd1e-b51b-4593-86bd-745985e440c2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:06.754553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 586/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:06.760346 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34995b8-9210-44d6-8cbc-beb2049ffdc9 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:06.855764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34995b8-9210-44d6-8cbc-beb2049ffdc9 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.877761 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:06.891367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa9817-db80-4e97-a8ba-7d9a486f0a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.892245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa9817-db80-4e97-a8ba-7d9a486f0a93 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.892530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa9817-db80-4e97-a8ba-7d9a486f0a93 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.892807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa9817-db80-4e97-a8ba-7d9a486f0a93 None admin] PATH_INFO: `/v3/users/ba5b0caa6d8c4236912bea7c7b2483e5` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.894148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa9817-db80-4e97-a8ba-7d9a486f0a93 None admin] RBAC: Authorizing `identity:get_user(user_id=ba5b0caa6d8c4236912bea7c7b2483e5)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:06.895638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa9817-db80-4e97-a8ba-7d9a486f0a93 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:06.897695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 572/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:06 2022] GET /identity/v3/users/ba5b0caa6d8c4236912bea7c7b2483e5 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:06.903141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70dd1e-b51b-4593-86bd-745985e440c2 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:07.002748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70dd1e-b51b-4593-86bd-745985e440c2 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:07.036174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3656e8-fabf-4251-9f3b-e03cb2ee8fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:07.036827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3656e8-fabf-4251-9f3b-e03cb2ee8fd5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:07.037114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3656e8-fabf-4251-9f3b-e03cb2ee8fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:07.037394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3656e8-fabf-4251-9f3b-e03cb2ee8fd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:07.070081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:07.104015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3656e8-fabf-4251-9f3b-e03cb2ee8fd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:07.105458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3656e8-fabf-4251-9f3b-e03cb2ee8fd5 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:07.199121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 587/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:07.204907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa9817-db80-4e97-a8ba-7d9a486f0a93 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:07.300004 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa9817-db80-4e97-a8ba-7d9a486f0a93 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:07.331027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7992f95-36ed-4432-8094-445136daf28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:07.331875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7992f95-36ed-4432-8094-445136daf28d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:07.332157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7992f95-36ed-4432-8094-445136daf28d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:07.332427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7992f95-36ed-4432-8094-445136daf28d None admin] PATH_INFO: `/v3/projects/989513614f744c60ad2bc89a348c6871` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:07.333580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7992f95-36ed-4432-8094-445136daf28d None admin] RBAC: Authorizing `identity:get_project(project_id=989513614f744c60ad2bc89a348c6871)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:07.335018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7992f95-36ed-4432-8094-445136daf28d None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:07.337219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 573/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:07 2022] GET /identity/v3/projects/989513614f744c60ad2bc89a348c6871 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:07.344637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3656e8-fabf-4251-9f3b-e03cb2ee8fd5 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:07.408673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:07.441091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3656e8-fabf-4251-9f3b-e03cb2ee8fd5 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:07.474207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c06be4-068e-4bed-8b00-4f9bccf35dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:07.474815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06be4-068e-4bed-8b00-4f9bccf35dc3 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:07.475099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06be4-068e-4bed-8b00-4f9bccf35dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:07.475375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06be4-068e-4bed-8b00-4f9bccf35dc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:07.476199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c06be4-068e-4bed-8b00-4f9bccf35dc3 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=989513614f744c60ad2bc89a348c6871, user.id=ba5b0caa6d8c4236912bea7c7b2483e5) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:07.476531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c06be4-068e-4bed-8b00-4f9bccf35dc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:07.477949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c06be4-068e-4bed-8b00-4f9bccf35dc3 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:07.483621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 588/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:07 2022] GET /identity/v3/role_assignments?user.id=ba5b0caa6d8c4236912bea7c7b2483e5&scope.project.id=989513614f744c60ad2bc89a348c6871&role.id=7a833c45603a4830b888e58e1232712a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:09.533650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807b24d-ad2e-4eea-9054-d162cdda839a None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.534180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807b24d-ad2e-4eea-9054-d162cdda839a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.534180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807b24d-ad2e-4eea-9054-d162cdda839a None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.535609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 574/1162] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:09.540926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33a0f-459f-4e88-b2a7-b165466a96a5 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.541307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33a0f-459f-4e88-b2a7-b165466a96a5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.541477 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33a0f-459f-4e88-b2a7-b165466a96a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.558135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d2c33a0f-459f-4e88-b2a7-b165466a96a5 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:09.559341 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d2c33a0f-459f-4e88-b2a7-b165466a96a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:09.623271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 589/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:09.645455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807b24d-ad2e-4eea-9054-d162cdda839a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:09.705197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:09.745232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807b24d-ad2e-4eea-9054-d162cdda839a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:09.777624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d8663-5b18-4fdb-8b4f-9e6e81e31b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:09.778257 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d8663-5b18-4fdb-8b4f-9e6e81e31b3c None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.778560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d8663-5b18-4fdb-8b4f-9e6e81e31b3c None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.778831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d8663-5b18-4fdb-8b4f-9e6e81e31b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.842053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d8663-5b18-4fdb-8b4f-9e6e81e31b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:09.843590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d8663-5b18-4fdb-8b4f-9e6e81e31b3c None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:09.897822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:09.937557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 575/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:42:09.943495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c33a0f-459f-4e88-b2a7-b165466a96a5 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.000286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:10.041334 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c33a0f-459f-4e88-b2a7-b165466a96a5 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:10.074099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d34caa-7272-4ff0-9276-7c696f9835f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.074931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d34caa-7272-4ff0-9276-7c696f9835f0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.075215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d34caa-7272-4ff0-9276-7c696f9835f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.075487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d34caa-7272-4ff0-9276-7c696f9835f0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.081914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d34caa-7272-4ff0-9276-7c696f9835f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.083390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d34caa-7272-4ff0-9276-7c696f9835f0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-c7d34caa-7272-4ff0-9276-7c696f9835f0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:10.097716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:10.098947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:42:10.100187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 590/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 05:42:09 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:10.104947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d8663-5b18-4fdb-8b4f-9e6e81e31b3c None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.199383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d8663-5b18-4fdb-8b4f-9e6e81e31b3c None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:10.230966 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7154c98-e1f1-45d2-845d-e98f1dd29063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.231608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7154c98-e1f1-45d2-845d-e98f1dd29063 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.231884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7154c98-e1f1-45d2-845d-e98f1dd29063 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.232154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7154c98-e1f1-45d2-845d-e98f1dd29063 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.232857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7154c98-e1f1-45d2-845d-e98f1dd29063 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:10.233158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7154c98-e1f1-45d2-845d-e98f1dd29063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.234508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7154c98-e1f1-45d2-845d-e98f1dd29063 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.241329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 576/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Jul 25 05:42:10 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:12.235344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810b1e7-15a1-40d7-8704-7bd956639a32 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.235938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810b1e7-15a1-40d7-8704-7bd956639a32 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.235938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810b1e7-15a1-40d7-8704-7bd956639a32 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.237189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 591/1167] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:12.242287 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cd361-1d56-44db-bfef-f5eb67dc4915 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.242758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cd361-1d56-44db-bfef-f5eb67dc4915 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.242860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cd361-1d56-44db-bfef-f5eb67dc4915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.259719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8f9cd361-1d56-44db-bfef-f5eb67dc4915 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:12.260900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8f9cd361-1d56-44db-bfef-f5eb67dc4915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:12.278354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:12.326045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 577/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:12.348478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810b1e7-15a1-40d7-8704-7bd956639a32 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:12.445873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810b1e7-15a1-40d7-8704-7bd956639a32 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:12.465629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:12.479436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87064718-6935-4d28-bc39-caf0954aa98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:12.480079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87064718-6935-4d28-bc39-caf0954aa98b None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.480360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87064718-6935-4d28-bc39-caf0954aa98b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.480637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87064718-6935-4d28-bc39-caf0954aa98b None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.481382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87064718-6935-4d28-bc39-caf0954aa98b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:12.482779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87064718-6935-4d28-bc39-caf0954aa98b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:12.544905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 592/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Mon Jul 25 05:42:12 2022] POST /identity/v3/domains => generated 267 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:42:14.583983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f27172-657c-4f0e-8ce8-531ca52782b4 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.584511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f27172-657c-4f0e-8ce8-531ca52782b4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.584511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f27172-657c-4f0e-8ce8-531ca52782b4 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.585947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 578/1170] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:14.591128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe479-7aa8-4228-8dfb-dea32a3dd01b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.591512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe479-7aa8-4228-8dfb-dea32a3dd01b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.591681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe479-7aa8-4228-8dfb-dea32a3dd01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.608184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-463fe479-7aa8-4228-8dfb-dea32a3dd01b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:14.609362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-463fe479-7aa8-4228-8dfb-dea32a3dd01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:14.673922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 593/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:14.697856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f27172-657c-4f0e-8ce8-531ca52782b4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:14.795128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f27172-657c-4f0e-8ce8-531ca52782b4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:14.826670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057ba55-c779-4162-ad16-768b8e5330ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:14.827511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057ba55-c779-4162-ad16-768b8e5330ef None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.827788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057ba55-c779-4162-ad16-768b8e5330ef None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.828057 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057ba55-c779-4162-ad16-768b8e5330ef None admin] PATH_INFO: `/v3/domains/9ed08a3fd9d04a0381d07921e1de16a8` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.838463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057ba55-c779-4162-ad16-768b8e5330ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=9ed08a3fd9d04a0381d07921e1de16a8)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:14.840021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057ba55-c779-4162-ad16-768b8e5330ef None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:14.841626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 579/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Mon Jul 25 05:42:14 2022] GET /identity/v3/domains/9ed08a3fd9d04a0381d07921e1de16a8 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:14.847321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fe479-7aa8-4228-8dfb-dea32a3dd01b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:14.942759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fe479-7aa8-4228-8dfb-dea32a3dd01b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:14.953371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:14.976008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4fb83-55c2-426d-a2ab-5cbf88e6d6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:14.976697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fb83-55c2-426d-a2ab-5cbf88e6d6f6 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.976982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fb83-55c2-426d-a2ab-5cbf88e6d6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.977262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fb83-55c2-426d-a2ab-5cbf88e6d6f6 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.977968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4fb83-55c2-426d-a2ab-5cbf88e6d6f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:14.979433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4fb83-55c2-426d-a2ab-5cbf88e6d6f6 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:15.064547 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 594/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:42:14 2022] POST /identity/v3/projects => generated 361 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:17.085030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac440fa-d94d-454d-bfee-c2fae4e4b4ef None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.085612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac440fa-d94d-454d-bfee-c2fae4e4b4ef None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.085612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac440fa-d94d-454d-bfee-c2fae4e4b4ef None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.087150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 580/1174] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:42:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:17.092241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8f4bd-c14c-4333-9559-c1c9ca302849 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.092637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8f4bd-c14c-4333-9559-c1c9ca302849 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.092793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8f4bd-c14c-4333-9559-c1c9ca302849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.108966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-11f8f4bd-c14c-4333-9559-c1c9ca302849 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:17.110138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-11f8f4bd-c14c-4333-9559-c1c9ca302849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:17.173678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 595/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:17.197706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac440fa-d94d-454d-bfee-c2fae4e4b4ef None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:17.213308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:17.295381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac440fa-d94d-454d-bfee-c2fae4e4b4ef None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:17.332282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510962e-b73c-4208-8227-47858569cffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:17.333129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510962e-b73c-4208-8227-47858569cffd None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.333410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510962e-b73c-4208-8227-47858569cffd None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.333683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510962e-b73c-4208-8227-47858569cffd None admin] PATH_INFO: `/v3/domains/9ed08a3fd9d04a0381d07921e1de16a8` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.334860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510962e-b73c-4208-8227-47858569cffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=9ed08a3fd9d04a0381d07921e1de16a8)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:17.336295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510962e-b73c-4208-8227-47858569cffd None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:17.337881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 581/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Mon Jul 25 05:42:17 2022] GET /identity/v3/domains/9ed08a3fd9d04a0381d07921e1de16a8 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:17.343516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8f4bd-c14c-4333-9559-c1c9ca302849 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:17.428995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:17.437094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8f4bd-c14c-4333-9559-c1c9ca302849 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:17.469611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce27d1c-cbe3-4787-af8a-f0f832db3b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:17.470304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce27d1c-cbe3-4787-af8a-f0f832db3b17 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.470610 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce27d1c-cbe3-4787-af8a-f0f832db3b17 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.470883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce27d1c-cbe3-4787-af8a-f0f832db3b17 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.471589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce27d1c-cbe3-4787-af8a-f0f832db3b17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:17.473004 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce27d1c-cbe3-4787-af8a-f0f832db3b17 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:17.565263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 596/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:42:17 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:19.614922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe843d-41d4-4f5f-b4c4-485131b9b931 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:19.615475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe843d-41d4-4f5f-b4c4-485131b9b931 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:19.615475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe843d-41d4-4f5f-b4c4-485131b9b931 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:19.616834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 582/1178] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:19.622208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034bad2-1675-41b8-b0af-d89911eaa9e0 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:19.622656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034bad2-1675-41b8-b0af-d89911eaa9e0 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:19.622800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034bad2-1675-41b8-b0af-d89911eaa9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:19.639850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e034bad2-1675-41b8-b0af-d89911eaa9e0 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:19.641033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e034bad2-1675-41b8-b0af-d89911eaa9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:19.704313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 597/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:19.727909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe843d-41d4-4f5f-b4c4-485131b9b931 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:19.808130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:19.826753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe843d-41d4-4f5f-b4c4-485131b9b931 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:19.860189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279ce8b-cb58-44cf-97a8-b59a602b6312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:19.860972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279ce8b-cb58-44cf-97a8-b59a602b6312 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:19.861251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279ce8b-cb58-44cf-97a8-b59a602b6312 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:19.861522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279ce8b-cb58-44cf-97a8-b59a602b6312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:19.872117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279ce8b-cb58-44cf-97a8-b59a602b6312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-2279ce8b-cb58-44cf-97a8-b59a602b6312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:19.879723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:19.880904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:42:19.881963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 583/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:19.887404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034bad2-1675-41b8-b0af-d89911eaa9e0 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:19.960577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:19.980227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034bad2-1675-41b8-b0af-d89911eaa9e0 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:20.013098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a2b4f-31bc-46ac-a971-4558810d180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:20.014014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a2b4f-31bc-46ac-a971-4558810d180c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.014300 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a2b4f-31bc-46ac-a971-4558810d180c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.014832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a2b4f-31bc-46ac-a971-4558810d180c None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.016161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a2b4f-31bc-46ac-a971-4558810d180c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:20.016474 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a2b4f-31bc-46ac-a971-4558810d180c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:20.017850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a2b4f-31bc-46ac-a971-4558810d180c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:20.026439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 598/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:20.032292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279ce8b-cb58-44cf-97a8-b59a602b6312 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:20.129911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279ce8b-cb58-44cf-97a8-b59a602b6312 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:20.163695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4321917-0f3a-4dad-a9ad-4df298d3edf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:20.165074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4321917-0f3a-4dad-a9ad-4df298d3edf1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.165353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4321917-0f3a-4dad-a9ad-4df298d3edf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.165626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4321917-0f3a-4dad-a9ad-4df298d3edf1 None admin] PATH_INFO: `/v3/users/a6c4ef445bfb4b479eb1d61c3de7ffda` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.176765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:20.179165 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4321917-0f3a-4dad-a9ad-4df298d3edf1 None admin] RBAC: Authorizing `identity:get_user(user_id=a6c4ef445bfb4b479eb1d61c3de7ffda)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:20.181240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4321917-0f3a-4dad-a9ad-4df298d3edf1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:20.183022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 584/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:20 2022] GET /identity/v3/users/a6c4ef445bfb4b479eb1d61c3de7ffda => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:20.188767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a2b4f-31bc-46ac-a971-4558810d180c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:20.284599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a2b4f-31bc-46ac-a971-4558810d180c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:20.317253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fb95a-9b54-493c-8153-0c47766c0e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:20.317888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb95a-9b54-493c-8153-0c47766c0e25 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.318174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb95a-9b54-493c-8153-0c47766c0e25 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.318454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb95a-9b54-493c-8153-0c47766c0e25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.329015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:20.386328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fb95a-9b54-493c-8153-0c47766c0e25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:20.387727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fb95a-9b54-493c-8153-0c47766c0e25 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:20.483414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 599/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:20.489336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4321917-0f3a-4dad-a9ad-4df298d3edf1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:20.625881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4321917-0f3a-4dad-a9ad-4df298d3edf1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:20.659065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba962510-d1c9-4492-a161-25da8d684e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:20.659860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba962510-d1c9-4492-a161-25da8d684e54 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.660136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba962510-d1c9-4492-a161-25da8d684e54 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.660407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba962510-d1c9-4492-a161-25da8d684e54 None admin] PATH_INFO: `/v3/projects/cbcc1ebf866e4928ad70ea142133ff2f` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.661695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba962510-d1c9-4492-a161-25da8d684e54 None admin] RBAC: Authorizing `identity:get_project(project_id=cbcc1ebf866e4928ad70ea142133ff2f)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:20.663165 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba962510-d1c9-4492-a161-25da8d684e54 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:20.665474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 585/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:20 2022] GET /identity/v3/projects/cbcc1ebf866e4928ad70ea142133ff2f => generated 361 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:20.671451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fb95a-9b54-493c-8153-0c47766c0e25 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:20.716403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:20.771689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fb95a-9b54-493c-8153-0c47766c0e25 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:20.804417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543d1c0-86e9-4da1-a38e-7093cb42211e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:20.805096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543d1c0-86e9-4da1-a38e-7093cb42211e None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.805376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543d1c0-86e9-4da1-a38e-7093cb42211e None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.805650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543d1c0-86e9-4da1-a38e-7093cb42211e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.806497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543d1c0-86e9-4da1-a38e-7093cb42211e None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=cbcc1ebf866e4928ad70ea142133ff2f, user.id=a6c4ef445bfb4b479eb1d61c3de7ffda) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:20.806810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543d1c0-86e9-4da1-a38e-7093cb42211e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:20.808220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543d1c0-86e9-4da1-a38e-7093cb42211e None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:20.813449 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 600/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:20 2022] GET /identity/v3/role_assignments?user.id=a6c4ef445bfb4b479eb1d61c3de7ffda&scope.project.id=cbcc1ebf866e4928ad70ea142133ff2f&role.id=7a833c45603a4830b888e58e1232712a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:22.790220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea35b5-1516-4b9e-914d-cd67012ed5ab None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:22.790823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea35b5-1516-4b9e-914d-cd67012ed5ab None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:22.790823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea35b5-1516-4b9e-914d-cd67012ed5ab None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:22.792238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 586/1186] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:22.797395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b897ccb-d142-43d9-a387-f86d68eced1e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:22.797824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b897ccb-d142-43d9-a387-f86d68eced1e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:22.797932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b897ccb-d142-43d9-a387-f86d68eced1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:22.814156 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-2b897ccb-d142-43d9-a387-f86d68eced1e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:22.815385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2b897ccb-d142-43d9-a387-f86d68eced1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:22.878205 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 601/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:22.900838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea35b5-1516-4b9e-914d-cd67012ed5ab None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:22.967371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.000398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea35b5-1516-4b9e-914d-cd67012ed5ab None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.033805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca2ea-0961-46f6-a9fc-7e655da91f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.034611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca2ea-0961-46f6-a9fc-7e655da91f22 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.034889 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca2ea-0961-46f6-a9fc-7e655da91f22 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.035159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca2ea-0961-46f6-a9fc-7e655da91f22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.045832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ca2ea-0961-46f6-a9fc-7e655da91f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-f17ca2ea-0961-46f6-a9fc-7e655da91f22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:23.053837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:23.055016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:42:23.056633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 587/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:23.063905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b897ccb-d142-43d9-a387-f86d68eced1e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.100836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.165799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b897ccb-d142-43d9-a387-f86d68eced1e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.198084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f772b60-3813-47f4-9111-7071c05279ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.198764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f772b60-3813-47f4-9111-7071c05279ca None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.199047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f772b60-3813-47f4-9111-7071c05279ca None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.199321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f772b60-3813-47f4-9111-7071c05279ca None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.200065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f772b60-3813-47f4-9111-7071c05279ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:23.200369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f772b60-3813-47f4-9111-7071c05279ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.201741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f772b60-3813-47f4-9111-7071c05279ca None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.209498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 602/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:23.216325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca2ea-0961-46f6-a9fc-7e655da91f22 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.313884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca2ea-0961-46f6-a9fc-7e655da91f22 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.339951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.347608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54b263-9d9d-42e9-b8f5-0f4665be3f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.353246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54b263-9d9d-42e9-b8f5-0f4665be3f30 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.353246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54b263-9d9d-42e9-b8f5-0f4665be3f30 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.353246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54b263-9d9d-42e9-b8f5-0f4665be3f30 None admin] PATH_INFO: `/v3/users/a6c4ef445bfb4b479eb1d61c3de7ffda` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.353246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed54b263-9d9d-42e9-b8f5-0f4665be3f30 None admin] RBAC: Authorizing `identity:get_user(user_id=a6c4ef445bfb4b479eb1d61c3de7ffda)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.353246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed54b263-9d9d-42e9-b8f5-0f4665be3f30 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.353673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 588/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/users/a6c4ef445bfb4b479eb1d61c3de7ffda => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:23.359147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f772b60-3813-47f4-9111-7071c05279ca None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.448426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.454294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f772b60-3813-47f4-9111-7071c05279ca None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.488328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d519c0f-f40a-42dc-a180-ca00618e52ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.489165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d519c0f-f40a-42dc-a180-ca00618e52ae None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.489444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d519c0f-f40a-42dc-a180-ca00618e52ae None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.489717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d519c0f-f40a-42dc-a180-ca00618e52ae None admin] PATH_INFO: `/v3/projects/cbcc1ebf866e4928ad70ea142133ff2f` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.490949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d519c0f-f40a-42dc-a180-ca00618e52ae None admin] RBAC: Authorizing `identity:get_project(project_id=cbcc1ebf866e4928ad70ea142133ff2f)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.492883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d519c0f-f40a-42dc-a180-ca00618e52ae None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.494627 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 603/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/projects/cbcc1ebf866e4928ad70ea142133ff2f => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:23.500447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54b263-9d9d-42e9-b8f5-0f4665be3f30 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.601769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54b263-9d9d-42e9-b8f5-0f4665be3f30 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.635395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aaf9ec-61ba-4eaf-a370-172b21b90329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.636271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaf9ec-61ba-4eaf-a370-172b21b90329 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.636550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaf9ec-61ba-4eaf-a370-172b21b90329 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.636822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaf9ec-61ba-4eaf-a370-172b21b90329 None admin] PATH_INFO: `/v3/projects/cbcc1ebf866e4928ad70ea142133ff2f/users/a6c4ef445bfb4b479eb1d61c3de7ffda/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.639018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aaf9ec-61ba-4eaf-a370-172b21b90329 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbcc1ebf866e4928ad70ea142133ff2f, user_id=a6c4ef445bfb4b479eb1d61c3de7ffda, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.640457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aaf9ec-61ba-4eaf-a370-172b21b90329 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.702791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 589/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:42:23 2022] PUT /identity/v3/projects/cbcc1ebf866e4928ad70ea142133ff2f/users/a6c4ef445bfb4b479eb1d61c3de7ffda/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:25.692709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe62eb-77de-435d-bda1-e14497b25f87 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:25.693285 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe62eb-77de-435d-bda1-e14497b25f87 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:25.693382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe62eb-77de-435d-bda1-e14497b25f87 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:25.694780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 604/1193] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:25.700603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18283e14-921d-4db4-a8dc-8980a505a3f3 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:25.701004 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18283e14-921d-4db4-a8dc-8980a505a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:25.701155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18283e14-921d-4db4-a8dc-8980a505a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:25.719120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-18283e14-921d-4db4-a8dc-8980a505a3f3 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:25.720291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-18283e14-921d-4db4-a8dc-8980a505a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:25.769558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:25.785184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 590/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:25.808684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe62eb-77de-435d-bda1-e14497b25f87 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:25.906080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe62eb-77de-435d-bda1-e14497b25f87 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:25.938417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4b3b0-db4f-4594-99bd-8fbf91252690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:25.939283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4b3b0-db4f-4594-99bd-8fbf91252690 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:25.939560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4b3b0-db4f-4594-99bd-8fbf91252690 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:25.939831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4b3b0-db4f-4594-99bd-8fbf91252690 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:25.950215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4b3b0-db4f-4594-99bd-8fbf91252690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-68d4b3b0-db4f-4594-99bd-8fbf91252690 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:25.957715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:25.958858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:42:25.959902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 605/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:25.964968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18283e14-921d-4db4-a8dc-8980a505a3f3 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:26.060991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18283e14-921d-4db4-a8dc-8980a505a3f3 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:26.093626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811aa479-ca22-4906-8fd6-1dd221d8b09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:26.094304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aa479-ca22-4906-8fd6-1dd221d8b09a None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:26.094618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aa479-ca22-4906-8fd6-1dd221d8b09a None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:26.094888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811aa479-ca22-4906-8fd6-1dd221d8b09a None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:26.095616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811aa479-ca22-4906-8fd6-1dd221d8b09a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:26.095946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811aa479-ca22-4906-8fd6-1dd221d8b09a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:26.097309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811aa479-ca22-4906-8fd6-1dd221d8b09a None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:26.105275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 591/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:26.110896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4b3b0-db4f-4594-99bd-8fbf91252690 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:26.136657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:26.207561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4b3b0-db4f-4594-99bd-8fbf91252690 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:26.239916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1a001-a8a1-4b15-bf5f-ebd07034f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:26.240773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1a001-a8a1-4b15-bf5f-ebd07034f00a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:26.241051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1a001-a8a1-4b15-bf5f-ebd07034f00a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:26.241347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1a001-a8a1-4b15-bf5f-ebd07034f00a None admin] PATH_INFO: `/v3/users/a6c4ef445bfb4b479eb1d61c3de7ffda` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:26.243131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1a001-a8a1-4b15-bf5f-ebd07034f00a None admin] RBAC: Authorizing `identity:get_user(user_id=a6c4ef445bfb4b479eb1d61c3de7ffda)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:26.244556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1a001-a8a1-4b15-bf5f-ebd07034f00a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:26.246154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 606/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:26 2022] GET /identity/v3/users/a6c4ef445bfb4b479eb1d61c3de7ffda => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:26.251882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811aa479-ca22-4906-8fd6-1dd221d8b09a None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:26.300229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:26.349839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811aa479-ca22-4906-8fd6-1dd221d8b09a None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:26.383210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012c928-6157-401d-b364-fa0cfa28c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:26.383843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012c928-6157-401d-b364-fa0cfa28c106 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:26.384120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012c928-6157-401d-b364-fa0cfa28c106 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:26.384392 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012c928-6157-401d-b364-fa0cfa28c106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:26.449034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012c928-6157-401d-b364-fa0cfa28c106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:26.450486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012c928-6157-401d-b364-fa0cfa28c106 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:26.496035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:26.546288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 592/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:26.552209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1a001-a8a1-4b15-bf5f-ebd07034f00a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:26.639331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:26.647674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1a001-a8a1-4b15-bf5f-ebd07034f00a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:26.681218 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ad514-c052-4a90-8070-e098dd887a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:26.682061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ad514-c052-4a90-8070-e098dd887a9b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:26.682341 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ad514-c052-4a90-8070-e098dd887a9b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:26.682650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ad514-c052-4a90-8070-e098dd887a9b None admin] PATH_INFO: `/v3/projects/cbcc1ebf866e4928ad70ea142133ff2f` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:26.683815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ad514-c052-4a90-8070-e098dd887a9b None admin] RBAC: Authorizing `identity:get_project(project_id=cbcc1ebf866e4928ad70ea142133ff2f)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:26.685249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ad514-c052-4a90-8070-e098dd887a9b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:26.686971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 607/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:26 2022] GET /identity/v3/projects/cbcc1ebf866e4928ad70ea142133ff2f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:26.692934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012c928-6157-401d-b364-fa0cfa28c106 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:26.787404 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012c928-6157-401d-b364-fa0cfa28c106 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:26.819882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a724d2-6d68-4a29-9e94-03e7903a2c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:26.820557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a724d2-6d68-4a29-9e94-03e7903a2c84 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:26.820835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a724d2-6d68-4a29-9e94-03e7903a2c84 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:26.821105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a724d2-6d68-4a29-9e94-03e7903a2c84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:26.821913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a724d2-6d68-4a29-9e94-03e7903a2c84 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=cbcc1ebf866e4928ad70ea142133ff2f, user.id=a6c4ef445bfb4b479eb1d61c3de7ffda) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:26.822219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a724d2-6d68-4a29-9e94-03e7903a2c84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:26.823650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a724d2-6d68-4a29-9e94-03e7903a2c84 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:26.829221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 593/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:26 2022] GET /identity/v3/role_assignments?user.id=a6c4ef445bfb4b479eb1d61c3de7ffda&scope.project.id=cbcc1ebf866e4928ad70ea142133ff2f&role.id=7a833c45603a4830b888e58e1232712a => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:36.677119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=20 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:36.826741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=94 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:36.108586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d23d7-0868-4db0-935a-4809564c1b35 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:36.109190 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d23d7-0868-4db0-935a-4809564c1b35 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:36.109190 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d23d7-0868-4db0-935a-4809564c1b35 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:36.110542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 608/1201] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:36.115788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b11c0-b4f0-45b5-a39f-99347999474b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:36.116211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b11c0-b4f0-45b5-a39f-99347999474b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:36.116464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b11c0-b4f0-45b5-a39f-99347999474b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:36.133517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-226b11c0-b4f0-45b5-a39f-99347999474b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:36.134693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-226b11c0-b4f0-45b5-a39f-99347999474b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:36.201604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 594/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:36.225898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d23d7-0868-4db0-935a-4809564c1b35 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:36.324915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d23d7-0868-4db0-935a-4809564c1b35 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:36.370317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc038fb-d846-4a57-a79a-7ac4d18ab1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:36.371358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc038fb-d846-4a57-a79a-7ac4d18ab1b1 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:36.371736 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc038fb-d846-4a57-a79a-7ac4d18ab1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:36.372133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc038fb-d846-4a57-a79a-7ac4d18ab1b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:36.382587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc038fb-d846-4a57-a79a-7ac4d18ab1b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:36.384055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc038fb-d846-4a57-a79a-7ac4d18ab1b1 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-cdc038fb-d846-4a57-a79a-7ac4d18ab1b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:36.402039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:36.403269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:36.404678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:36.404678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:36.404678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:36.404678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:36.404678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 609/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:36.409384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b11c0-b4f0-45b5-a39f-99347999474b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:36.509841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b11c0-b4f0-45b5-a39f-99347999474b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:36.537730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:36.544416 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109f627-d483-4a83-a8de-26b6b20e4c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:36.545153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109f627-d483-4a83-a8de-26b6b20e4c2b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:36.545501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109f627-d483-4a83-a8de-26b6b20e4c2b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:36.545835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109f627-d483-4a83-a8de-26b6b20e4c2b None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:36.546590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109f627-d483-4a83-a8de-26b6b20e4c2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:36.549338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109f627-d483-4a83-a8de-26b6b20e4c2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:36.549338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109f627-d483-4a83-a8de-26b6b20e4c2b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:36.559266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 595/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:36.565269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc038fb-d846-4a57-a79a-7ac4d18ab1b1 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:36.616942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:36.662420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc038fb-d846-4a57-a79a-7ac4d18ab1b1 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:36.694900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197aae3-026c-4ede-9945-f3ac59290b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:36.695553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197aae3-026c-4ede-9945-f3ac59290b0c None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:36.695830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197aae3-026c-4ede-9945-f3ac59290b0c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:36.696101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197aae3-026c-4ede-9945-f3ac59290b0c None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:36.696800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197aae3-026c-4ede-9945-f3ac59290b0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:36.698245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197aae3-026c-4ede-9945-f3ac59290b0c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:36.764220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 610/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Jul 25 05:43:36 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:43:38.799751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5e578-32bc-404a-8f4b-324411ba12fc None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:38.800281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5e578-32bc-404a-8f4b-324411ba12fc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:38.800281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5e578-32bc-404a-8f4b-324411ba12fc None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:38.801678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 596/1206] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:38.806964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb760f3-b29f-47f5-97ca-ea701496f602 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:38.807960 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb760f3-b29f-47f5-97ca-ea701496f602 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:38.808144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb760f3-b29f-47f5-97ca-ea701496f602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:38.825404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-0fb760f3-b29f-47f5-97ca-ea701496f602 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:38.826623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0fb760f3-b29f-47f5-97ca-ea701496f602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:38.894862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 611/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:38.918250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5e578-32bc-404a-8f4b-324411ba12fc None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:39.020568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5e578-32bc-404a-8f4b-324411ba12fc None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:39.055271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbcd2e-4c5f-4d9f-9eb9-425104bfcf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:39.055271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbcd2e-4c5f-4d9f-9eb9-425104bfcf21 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:39.055271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbcd2e-4c5f-4d9f-9eb9-425104bfcf21 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:39.055271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbcd2e-4c5f-4d9f-9eb9-425104bfcf21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:39.065817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbcd2e-4c5f-4d9f-9eb9-425104bfcf21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-a7fbcd2e-4c5f-4d9f-9eb9-425104bfcf21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:43:39.073647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:39.079488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:39.080633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 597/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:43:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:39.081704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb760f3-b29f-47f5-97ca-ea701496f602 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:39.122122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:39.189519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb760f3-b29f-47f5-97ca-ea701496f602 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:39.231017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42bda6-7ef6-47c8-85ea-cc534cb3acf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:39.231726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42bda6-7ef6-47c8-85ea-cc534cb3acf8 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:39.232006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42bda6-7ef6-47c8-85ea-cc534cb3acf8 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:39.232276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42bda6-7ef6-47c8-85ea-cc534cb3acf8 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:39.233019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42bda6-7ef6-47c8-85ea-cc534cb3acf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:39.233322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42bda6-7ef6-47c8-85ea-cc534cb3acf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:39.234713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42bda6-7ef6-47c8-85ea-cc534cb3acf8 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:39.246687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 612/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:43:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:39.252482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbcd2e-4c5f-4d9f-9eb9-425104bfcf21 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:39.305585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:39.365042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbcd2e-4c5f-4d9f-9eb9-425104bfcf21 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:39.400095 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79131f-7005-4132-abdc-1d557ca6b2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:39.401400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79131f-7005-4132-abdc-1d557ca6b2c8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:39.401680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79131f-7005-4132-abdc-1d557ca6b2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:39.401993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79131f-7005-4132-abdc-1d557ca6b2c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:39.412673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79131f-7005-4132-abdc-1d557ca6b2c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:39.414322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79131f-7005-4132-abdc-1d557ca6b2c8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-7c79131f-7005-4132-abdc-1d557ca6b2c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:39.432934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:39.433936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:39.435067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:39.435067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:39.435067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:39.435067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:39.435067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 598/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:39.440372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42bda6-7ef6-47c8-85ea-cc534cb3acf8 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:39.550406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:39.553878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42bda6-7ef6-47c8-85ea-cc534cb3acf8 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:39.591109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a4d4d-51c9-49ab-b479-0c091687103a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:39.591704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a4d4d-51c9-49ab-b479-0c091687103a None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:39.591985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a4d4d-51c9-49ab-b479-0c091687103a None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:39.592261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a4d4d-51c9-49ab-b479-0c091687103a None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:39.592990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a4d4d-51c9-49ab-b479-0c091687103a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:39.593298 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a4d4d-51c9-49ab-b479-0c091687103a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:39.594806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a4d4d-51c9-49ab-b479-0c091687103a None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:39.606530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 613/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:39.612620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79131f-7005-4132-abdc-1d557ca6b2c8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:39.728564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79131f-7005-4132-abdc-1d557ca6b2c8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:39.730382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:39.770438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fca67-cc00-4bf8-b3c5-7bb51cfaf018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:39.771266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fca67-cc00-4bf8-b3c5-7bb51cfaf018 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:39.771542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fca67-cc00-4bf8-b3c5-7bb51cfaf018 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:39.771844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fca67-cc00-4bf8-b3c5-7bb51cfaf018 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:39.779744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fca67-cc00-4bf8-b3c5-7bb51cfaf018 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:39.781169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fca67-cc00-4bf8-b3c5-7bb51cfaf018 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-bf0fca67-cc00-4bf8-b3c5-7bb51cfaf018 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:43:39.796621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:39.800960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:39.802186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 599/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:39.803836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a4d4d-51c9-49ab-b479-0c091687103a None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:39.899928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a4d4d-51c9-49ab-b479-0c091687103a None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:39.933489 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf672152-74aa-44cc-88a2-15b6afb2ec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:39.934144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf672152-74aa-44cc-88a2-15b6afb2ec86 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:39.934421 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf672152-74aa-44cc-88a2-15b6afb2ec86 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:39.938282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf672152-74aa-44cc-88a2-15b6afb2ec86 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:39.938282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf672152-74aa-44cc-88a2-15b6afb2ec86 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:39.938282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf672152-74aa-44cc-88a2-15b6afb2ec86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:39.938282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf672152-74aa-44cc-88a2-15b6afb2ec86 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:39.954097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 614/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 05:43:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:39.959822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fca67-cc00-4bf8-b3c5-7bb51cfaf018 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:40.062904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fca67-cc00-4bf8-b3c5-7bb51cfaf018 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:40.099608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ec556-53be-4d49-9901-67da191c02f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:40.100259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ec556-53be-4d49-9901-67da191c02f7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:40.100542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ec556-53be-4d49-9901-67da191c02f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:40.100821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ec556-53be-4d49-9901-67da191c02f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:40.130082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:40.167523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ec556-53be-4d49-9901-67da191c02f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:40.168938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ec556-53be-4d49-9901-67da191c02f7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:40.264714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 600/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:43:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:43:40.270797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf672152-74aa-44cc-88a2-15b6afb2ec86 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:40.289195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:40.368959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf672152-74aa-44cc-88a2-15b6afb2ec86 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:40.402070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c558acb-075e-4ee0-b198-9512c293d475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:40.402927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c558acb-075e-4ee0-b198-9512c293d475 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:40.403206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c558acb-075e-4ee0-b198-9512c293d475 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:40.403478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c558acb-075e-4ee0-b198-9512c293d475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:40.413035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c558acb-075e-4ee0-b198-9512c293d475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:40.414461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c558acb-075e-4ee0-b198-9512c293d475 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-3c558acb-075e-4ee0-b198-9512c293d475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:40.433130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:40.434206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:40.435329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:40.435329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:40.435329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:40.435329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:40.435329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 615/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:40.440040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ec556-53be-4d49-9901-67da191c02f7 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:40.499663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:40.536132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ec556-53be-4d49-9901-67da191c02f7 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:40.568287 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05883bf8-48bf-4d1a-81f1-f220203d7b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:40.568934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05883bf8-48bf-4d1a-81f1-f220203d7b35 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:40.569212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05883bf8-48bf-4d1a-81f1-f220203d7b35 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:40.569484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05883bf8-48bf-4d1a-81f1-f220203d7b35 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:40.570188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05883bf8-48bf-4d1a-81f1-f220203d7b35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:40.570508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05883bf8-48bf-4d1a-81f1-f220203d7b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:40.571858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05883bf8-48bf-4d1a-81f1-f220203d7b35 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:40.583728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 601/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:40.592753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c558acb-075e-4ee0-b198-9512c293d475 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:40.660271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:40.690642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c558acb-075e-4ee0-b198-9512c293d475 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:40.723017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a5451-15c5-4dae-83ff-46dfdfd9f046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:40.723970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a5451-15c5-4dae-83ff-46dfdfd9f046 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:40.724666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a5451-15c5-4dae-83ff-46dfdfd9f046 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:40.724941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a5451-15c5-4dae-83ff-46dfdfd9f046 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:40.731943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a5451-15c5-4dae-83ff-46dfdfd9f046 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:40.733446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a5451-15c5-4dae-83ff-46dfdfd9f046 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5d2a5451-15c5-4dae-83ff-46dfdfd9f046 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:43:40.748254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:40.749397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:40.750536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:43:40.750536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:43:40.750536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:43:40.750536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:43:40.750536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:43:40.750536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:43:40.750536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:43:40.750536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:40.750536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 616/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:43:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:40.755574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05883bf8-48bf-4d1a-81f1-f220203d7b35 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:40.853017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05883bf8-48bf-4d1a-81f1-f220203d7b35 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:40.875318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:40.886370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b159b0-93dc-4c70-b2f5-88c04c81d691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:40.887055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b159b0-93dc-4c70-b2f5-88c04c81d691 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:40.887337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b159b0-93dc-4c70-b2f5-88c04c81d691 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:40.887612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b159b0-93dc-4c70-b2f5-88c04c81d691 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:40.888389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b159b0-93dc-4c70-b2f5-88c04c81d691 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:40.888953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b159b0-93dc-4c70-b2f5-88c04c81d691 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:40.890383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b159b0-93dc-4c70-b2f5-88c04c81d691 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:40.902237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 602/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:40.908613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a5451-15c5-4dae-83ff-46dfdfd9f046 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:41.005632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a5451-15c5-4dae-83ff-46dfdfd9f046 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:41.033599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:41.039279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddb528-ade2-43a0-9351-a4a9804e1cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:41.039928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddb528-ade2-43a0-9351-a4a9804e1cab None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:41.040211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddb528-ade2-43a0-9351-a4a9804e1cab None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:41.040489 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddb528-ade2-43a0-9351-a4a9804e1cab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:41.041403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddb528-ade2-43a0-9351-a4a9804e1cab None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=dbfcf2e915c049f08a9e41c555401a6e) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:41.042220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddb528-ade2-43a0-9351-a4a9804e1cab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:41.043676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddb528-ade2-43a0-9351-a4a9804e1cab None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:41.048993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 617/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:43:40 2022] GET /identity/v3/role_assignments?user.id=dbfcf2e915c049f08a9e41c555401a6e&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:43.026623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e8da6-3342-43e9-b84b-3cff0bac448e None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:43.027189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e8da6-3342-43e9-b84b-3cff0bac448e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:43.027189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e8da6-3342-43e9-b84b-3cff0bac448e None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:43.028560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 603/1220] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:43.034085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba737a-d0a6-48ef-b988-801ea7a843b2 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:43.034464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba737a-d0a6-48ef-b988-801ea7a843b2 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:43.034692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba737a-d0a6-48ef-b988-801ea7a843b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:43.051209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-13ba737a-d0a6-48ef-b988-801ea7a843b2 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:43.052395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-13ba737a-d0a6-48ef-b988-801ea7a843b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:43.119087 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 618/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:43.143015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e8da6-3342-43e9-b84b-3cff0bac448e None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:43.242432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e8da6-3342-43e9-b84b-3cff0bac448e None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:43.276069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9528f0-27e9-4364-868f-f82ffa3c76a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:43.276857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9528f0-27e9-4364-868f-f82ffa3c76a3 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:43.277136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9528f0-27e9-4364-868f-f82ffa3c76a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:43.277409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9528f0-27e9-4364-868f-f82ffa3c76a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:43.287681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9528f0-27e9-4364-868f-f82ffa3c76a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-cf9528f0-27e9-4364-868f-f82ffa3c76a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:43:43.295947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:43.297077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:43.298136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 604/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:43:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:43.303386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba737a-d0a6-48ef-b988-801ea7a843b2 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:43.400552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba737a-d0a6-48ef-b988-801ea7a843b2 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:43.423782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:43.434741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694429-0001-44ac-a6cb-3a4d2b0e5066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:43.435412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694429-0001-44ac-a6cb-3a4d2b0e5066 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:43.435689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694429-0001-44ac-a6cb-3a4d2b0e5066 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:43.435963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694429-0001-44ac-a6cb-3a4d2b0e5066 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:43.436701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb694429-0001-44ac-a6cb-3a4d2b0e5066 None admin] RBAC: Adding query filter params (name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:43.437008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb694429-0001-44ac-a6cb-3a4d2b0e5066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:43.438385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb694429-0001-44ac-a6cb-3a4d2b0e5066 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:43.446677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 619/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:43:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:43.452530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9528f0-27e9-4364-868f-f82ffa3c76a3 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:43.499570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:43.554169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9528f0-27e9-4364-868f-f82ffa3c76a3 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:43.588306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a67811-6ed6-424b-928c-d89cf4095cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:43.589093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a67811-6ed6-424b-928c-d89cf4095cc4 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:43.589372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a67811-6ed6-424b-928c-d89cf4095cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:43.589643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a67811-6ed6-424b-928c-d89cf4095cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:43.600013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a67811-6ed6-424b-928c-d89cf4095cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:43.601439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a67811-6ed6-424b-928c-d89cf4095cc4 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-e3a67811-6ed6-424b-928c-d89cf4095cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:43.618753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:43.620343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:43.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:43.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:43.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:43.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:43.621333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 605/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:43.626541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694429-0001-44ac-a6cb-3a4d2b0e5066 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:43.725087 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694429-0001-44ac-a6cb-3a4d2b0e5066 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:43.758008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beedc7-e539-4c11-9351-4de36c31179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:43.758634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beedc7-e539-4c11-9351-4de36c31179c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:43.758916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beedc7-e539-4c11-9351-4de36c31179c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:43.759190 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beedc7-e539-4c11-9351-4de36c31179c None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:43.760058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9beedc7-e539-4c11-9351-4de36c31179c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:43.760369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9beedc7-e539-4c11-9351-4de36c31179c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:43.761796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9beedc7-e539-4c11-9351-4de36c31179c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:43.774230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 620/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:43.780659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a67811-6ed6-424b-928c-d89cf4095cc4 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:43.872345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:43.878820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a67811-6ed6-424b-928c-d89cf4095cc4 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:43.913125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4e3ed-627e-42b7-b085-b1f41ab529b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:43.914842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4e3ed-627e-42b7-b085-b1f41ab529b7 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:43.914842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4e3ed-627e-42b7-b085-b1f41ab529b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:43.914842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4e3ed-627e-42b7-b085-b1f41ab529b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:43.922046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4e3ed-627e-42b7-b085-b1f41ab529b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:43.923504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4e3ed-627e-42b7-b085-b1f41ab529b7 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-12b4e3ed-627e-42b7-b085-b1f41ab529b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:43:43.937965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:43.939132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:43.940168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 606/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:43.945331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beedc7-e539-4c11-9351-4de36c31179c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:43.993324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:44.045046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9beedc7-e539-4c11-9351-4de36c31179c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:44.078713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c87c8-82ca-479c-a918-95aa5c58a7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:44.079367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c87c8-82ca-479c-a918-95aa5c58a7b6 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:44.079648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c87c8-82ca-479c-a918-95aa5c58a7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:44.079923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c87c8-82ca-479c-a918-95aa5c58a7b6 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:44.080744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c87c8-82ca-479c-a918-95aa5c58a7b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:44.081083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c87c8-82ca-479c-a918-95aa5c58a7b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:44.082682 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c87c8-82ca-479c-a918-95aa5c58a7b6 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:44.099694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 621/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 05:43:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:44.105372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4e3ed-627e-42b7-b085-b1f41ab529b7 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:44.203273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4e3ed-627e-42b7-b085-b1f41ab529b7 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:44.235934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcc20c-e5e3-4d35-aa39-bbaa4d257973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:44.236718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcc20c-e5e3-4d35-aa39-bbaa4d257973 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:44.236996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcc20c-e5e3-4d35-aa39-bbaa4d257973 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:44.237265 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcc20c-e5e3-4d35-aa39-bbaa4d257973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:44.246930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fcc20c-e5e3-4d35-aa39-bbaa4d257973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:44.248526 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fcc20c-e5e3-4d35-aa39-bbaa4d257973 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-31fcc20c-e5e3-4d35-aa39-bbaa4d257973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:44.266316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:44.267438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:44.268810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:44.268810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:44.268810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:44.268810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:44.268810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 607/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:44.273301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c87c8-82ca-479c-a918-95aa5c58a7b6 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:44.373983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c87c8-82ca-479c-a918-95aa5c58a7b6 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:44.376256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:44.407937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30523749-f66e-4fbd-ad26-dc665ebfd1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:44.408589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30523749-f66e-4fbd-ad26-dc665ebfd1f6 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:44.408866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30523749-f66e-4fbd-ad26-dc665ebfd1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:44.409137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30523749-f66e-4fbd-ad26-dc665ebfd1f6 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:44.409903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30523749-f66e-4fbd-ad26-dc665ebfd1f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:44.410206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30523749-f66e-4fbd-ad26-dc665ebfd1f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:44.411612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30523749-f66e-4fbd-ad26-dc665ebfd1f6 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:44.422909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 622/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:44.428491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcc20c-e5e3-4d35-aa39-bbaa4d257973 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:44.437601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:44.524896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcc20c-e5e3-4d35-aa39-bbaa4d257973 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:44.559602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115973d-b370-478a-8fe9-16c21f59515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:44.560390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115973d-b370-478a-8fe9-16c21f59515b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:44.560666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115973d-b370-478a-8fe9-16c21f59515b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:44.560939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115973d-b370-478a-8fe9-16c21f59515b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:44.567375 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115973d-b370-478a-8fe9-16c21f59515b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:44.571788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115973d-b370-478a-8fe9-16c21f59515b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-5115973d-b370-478a-8fe9-16c21f59515b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:43:44.582622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:44.583725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:44.584856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:43:44.584856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:43:44.584856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:43:44.584856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:43:44.584856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:43:44.584856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:43:44.584856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:43:44.584856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:44.584856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 608/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:43:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:43:44.590206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30523749-f66e-4fbd-ad26-dc665ebfd1f6 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:44.685983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30523749-f66e-4fbd-ad26-dc665ebfd1f6 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:44.718187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d0720-8800-4bbd-b321-7785f67b12b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:44.718856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d0720-8800-4bbd-b321-7785f67b12b0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:44.719137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d0720-8800-4bbd-b321-7785f67b12b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:44.719408 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d0720-8800-4bbd-b321-7785f67b12b0 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:44.720183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d0720-8800-4bbd-b321-7785f67b12b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:44.720490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d0720-8800-4bbd-b321-7785f67b12b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:44.721902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d0720-8800-4bbd-b321-7785f67b12b0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:44.734030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 623/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:43:44.740119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115973d-b370-478a-8fe9-16c21f59515b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:44.805822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:44.840282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115973d-b370-478a-8fe9-16c21f59515b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:44.873259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f188b4-1f8f-4abb-b591-4fb5c0b9a3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:44.874143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f188b4-1f8f-4abb-b591-4fb5c0b9a3d0 None admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:44.874423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f188b4-1f8f-4abb-b591-4fb5c0b9a3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:44.874788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f188b4-1f8f-4abb-b591-4fb5c0b9a3d0 None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/dbfcf2e915c049f08a9e41c555401a6e/roles/9cc15ca9e61f425cacb9c1a9f52073dd` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:44.889608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f188b4-1f8f-4abb-b591-4fb5c0b9a3d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=dbfcf2e915c049f08a9e41c555401a6e, role_id=9cc15ca9e61f425cacb9c1a9f52073dd)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:44.891082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f188b4-1f8f-4abb-b591-4fb5c0b9a3d0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:44.979989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 609/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:43:44 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/dbfcf2e915c049f08a9e41c555401a6e/roles/9cc15ca9e61f425cacb9c1a9f52073dd => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:43:46.989425 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f605b2f-6852-473e-a37d-49a02a826442 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:46.989945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f605b2f-6852-473e-a37d-49a02a826442 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:46.990064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f605b2f-6852-473e-a37d-49a02a826442 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:46.991459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 624/1233] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:46.996730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0486e2-069b-4a7b-b69e-b29ba2eeaf32 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:46.997157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0486e2-069b-4a7b-b69e-b29ba2eeaf32 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:46.997282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0486e2-069b-4a7b-b69e-b29ba2eeaf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:47.014414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-be0486e2-069b-4a7b-b69e-b29ba2eeaf32 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:47.015641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-be0486e2-069b-4a7b-b69e-b29ba2eeaf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:47.081886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 610/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:47.105599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f605b2f-6852-473e-a37d-49a02a826442 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:47.130570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:47.203102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f605b2f-6852-473e-a37d-49a02a826442 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:47.236375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d340d7d-8aa2-477a-baf8-0e451c61321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:47.237163 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d340d7d-8aa2-477a-baf8-0e451c61321f None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:47.237445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d340d7d-8aa2-477a-baf8-0e451c61321f None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:47.237718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d340d7d-8aa2-477a-baf8-0e451c61321f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:47.248203 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d340d7d-8aa2-477a-baf8-0e451c61321f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-5d340d7d-8aa2-477a-baf8-0e451c61321f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:43:47.255898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:47.257043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:47.258108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 625/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Jul 25 05:43:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:47.263144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0486e2-069b-4a7b-b69e-b29ba2eeaf32 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:47.308193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:47.362054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0486e2-069b-4a7b-b69e-b29ba2eeaf32 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:47.395681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307f168-cf03-4aef-bbcb-878a7c32ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:47.396417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307f168-cf03-4aef-bbcb-878a7c32ac82 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:47.396699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307f168-cf03-4aef-bbcb-878a7c32ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:47.396972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307f168-cf03-4aef-bbcb-878a7c32ac82 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:47.397701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307f168-cf03-4aef-bbcb-878a7c32ac82 None admin] RBAC: Adding query filter params (name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:47.398006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307f168-cf03-4aef-bbcb-878a7c32ac82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:47.399427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307f168-cf03-4aef-bbcb-878a7c32ac82 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:47.407908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 611/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Jul 25 05:43:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:47.413661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d340d7d-8aa2-477a-baf8-0e451c61321f None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:47.499875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:47.516839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d340d7d-8aa2-477a-baf8-0e451c61321f None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:47.551022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddad4dc-45cb-4509-8805-5271afd4cc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:47.551801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddad4dc-45cb-4509-8805-5271afd4cc0b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:47.552076 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddad4dc-45cb-4509-8805-5271afd4cc0b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:47.552344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddad4dc-45cb-4509-8805-5271afd4cc0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:47.562350 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddad4dc-45cb-4509-8805-5271afd4cc0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:47.563842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddad4dc-45cb-4509-8805-5271afd4cc0b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-0ddad4dc-45cb-4509-8805-5271afd4cc0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:47.580972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:47.582159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:47.583301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:47.583301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:47.583301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:47.583301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:47.583301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 626/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:47.588159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307f168-cf03-4aef-bbcb-878a7c32ac82 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:47.686208 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307f168-cf03-4aef-bbcb-878a7c32ac82 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:47.694658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:47.718592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011b9b1-9b5b-4a3f-a4dd-5542c62b49f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:47.719248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011b9b1-9b5b-4a3f-a4dd-5542c62b49f5 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:47.719525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011b9b1-9b5b-4a3f-a4dd-5542c62b49f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:47.719804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011b9b1-9b5b-4a3f-a4dd-5542c62b49f5 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:47.720512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011b9b1-9b5b-4a3f-a4dd-5542c62b49f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:47.720811 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011b9b1-9b5b-4a3f-a4dd-5542c62b49f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:47.722159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011b9b1-9b5b-4a3f-a4dd-5542c62b49f5 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:47.734461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 612/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:47.740086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddad4dc-45cb-4509-8805-5271afd4cc0b None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:47.836572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddad4dc-45cb-4509-8805-5271afd4cc0b None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:47.870624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ca434-02d6-4171-ae12-6a39e829be6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:47.871432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ca434-02d6-4171-ae12-6a39e829be6c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:47.871711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ca434-02d6-4171-ae12-6a39e829be6c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:47.871975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ca434-02d6-4171-ae12-6a39e829be6c None admin] PATH_INFO: `/v3/users/placement` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:47.874616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:47.878952 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ca434-02d6-4171-ae12-6a39e829be6c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:47.880412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ca434-02d6-4171-ae12-6a39e829be6c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-c63ca434-02d6-4171-ae12-6a39e829be6c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:43:47.895400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:47.896566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:47.897954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 627/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:47.902349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011b9b1-9b5b-4a3f-a4dd-5542c62b49f5 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:48.000507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011b9b1-9b5b-4a3f-a4dd-5542c62b49f5 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:48.034381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d8bdf-66d2-45bb-a3e8-59abd0a2e674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:48.035069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8bdf-66d2-45bb-a3e8-59abd0a2e674 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:48.035353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8bdf-66d2-45bb-a3e8-59abd0a2e674 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:48.035633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8bdf-66d2-45bb-a3e8-59abd0a2e674 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:48.036540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d8bdf-66d2-45bb-a3e8-59abd0a2e674 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:48.036841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d8bdf-66d2-45bb-a3e8-59abd0a2e674 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:48.038260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d8bdf-66d2-45bb-a3e8-59abd0a2e674 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:48.055552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 613/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 05:43:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:48.061846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ca434-02d6-4171-ae12-6a39e829be6c None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:48.158362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ca434-02d6-4171-ae12-6a39e829be6c None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:48.191501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caae0f-f002-42d1-898b-f8ce2397d1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:48.192135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caae0f-f002-42d1-898b-f8ce2397d1cb None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:48.192418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caae0f-f002-42d1-898b-f8ce2397d1cb None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:48.192694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caae0f-f002-42d1-898b-f8ce2397d1cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:48.251949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:48.258319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caae0f-f002-42d1-898b-f8ce2397d1cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:48.259884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caae0f-f002-42d1-898b-f8ce2397d1cb None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:48.356641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 628/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:43:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:43:48.362493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d8bdf-66d2-45bb-a3e8-59abd0a2e674 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:48.389803 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:48.461252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d8bdf-66d2-45bb-a3e8-59abd0a2e674 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:48.494996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931e99d-bf5c-4148-9b38-2636cd24cf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:48.495786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931e99d-bf5c-4148-9b38-2636cd24cf01 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:48.496066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931e99d-bf5c-4148-9b38-2636cd24cf01 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:48.496340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931e99d-bf5c-4148-9b38-2636cd24cf01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:48.506120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931e99d-bf5c-4148-9b38-2636cd24cf01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:48.522598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931e99d-bf5c-4148-9b38-2636cd24cf01 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-e931e99d-bf5c-4148-9b38-2636cd24cf01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:48.525365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:48.526344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:48.528496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:48.528496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:48.528496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:48.528496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:48.528496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 614/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:48.532835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caae0f-f002-42d1-898b-f8ce2397d1cb None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:48.624320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:48.640028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caae0f-f002-42d1-898b-f8ce2397d1cb None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:48.665104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb0006-7868-4de1-be36-072b70891e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:48.665801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb0006-7868-4de1-be36-072b70891e30 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:48.666078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb0006-7868-4de1-be36-072b70891e30 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:48.666346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb0006-7868-4de1-be36-072b70891e30 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:48.667091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb0006-7868-4de1-be36-072b70891e30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:48.667392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb0006-7868-4de1-be36-072b70891e30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:48.668797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb0006-7868-4de1-be36-072b70891e30 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:48.679494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 629/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:48.685267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931e99d-bf5c-4148-9b38-2636cd24cf01 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:48.762700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:48.784085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931e99d-bf5c-4148-9b38-2636cd24cf01 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:48.816128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a42d2-149e-47c7-9364-1bbc5e438798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:48.816919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a42d2-149e-47c7-9364-1bbc5e438798 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:48.817198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a42d2-149e-47c7-9364-1bbc5e438798 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:48.817469 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a42d2-149e-47c7-9364-1bbc5e438798 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:48.824002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a42d2-149e-47c7-9364-1bbc5e438798 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:48.825957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a42d2-149e-47c7-9364-1bbc5e438798 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-b61a42d2-149e-47c7-9364-1bbc5e438798 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:43:48.839374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:48.843409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:48.844629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:43:48.844629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:43:48.844629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:43:48.844629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:43:48.844629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:43:48.844629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:43:48.844629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:43:48.844629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:48.844629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 615/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:43:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:48.846976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb0006-7868-4de1-be36-072b70891e30 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:48.946274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb0006-7868-4de1-be36-072b70891e30 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:48.979329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36337b5-0746-457e-9014-a4fb25974afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:48.979985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36337b5-0746-457e-9014-a4fb25974afa None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:48.980268 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36337b5-0746-457e-9014-a4fb25974afa None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:48.980544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36337b5-0746-457e-9014-a4fb25974afa None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:48.981421 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36337b5-0746-457e-9014-a4fb25974afa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:48.981737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36337b5-0746-457e-9014-a4fb25974afa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:48.983200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36337b5-0746-457e-9014-a4fb25974afa None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:48.994964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 630/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:43:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:49.001481 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a42d2-149e-47c7-9364-1bbc5e438798 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:49.100236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a42d2-149e-47c7-9364-1bbc5e438798 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:49.139259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601ed7e-4d34-43d0-adf6-93a71141e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:49.139898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601ed7e-4d34-43d0-adf6-93a71141e283 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:49.140177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601ed7e-4d34-43d0-adf6-93a71141e283 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:49.140450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601ed7e-4d34-43d0-adf6-93a71141e283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:49.141264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601ed7e-4d34-43d0-adf6-93a71141e283 None admin] RBAC: Adding query filter params (role.id=9cc15ca9e61f425cacb9c1a9f52073dd, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=dbfcf2e915c049f08a9e41c555401a6e) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:49.141570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601ed7e-4d34-43d0-adf6-93a71141e283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:49.143007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601ed7e-4d34-43d0-adf6-93a71141e283 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:49.148785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 616/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:43:49 2022] GET /identity/v3/role_assignments?user.id=dbfcf2e915c049f08a9e41c555401a6e&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=9cc15ca9e61f425cacb9c1a9f52073dd => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:51.206889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e8970-5b93-47cf-bb87-a198ca1d5a6c None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:51.207455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e8970-5b93-47cf-bb87-a198ca1d5a6c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:51.207455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e8970-5b93-47cf-bb87-a198ca1d5a6c None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:51.208789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 631/1247] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:51.213872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25bc56-733d-461a-b434-1a61aca06ec6 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:51.217056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25bc56-733d-461a-b434-1a61aca06ec6 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:51.217056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25bc56-733d-461a-b434-1a61aca06ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:51.220195 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:51.232533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ff25bc56-733d-461a-b434-1a61aca06ec6 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:51.233702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ff25bc56-733d-461a-b434-1a61aca06ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:51.299938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 617/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:51.323386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e8970-5b93-47cf-bb87-a198ca1d5a6c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:51.341835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:51.424970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e8970-5b93-47cf-bb87-a198ca1d5a6c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:51.457800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2245ec-bfb3-406b-b6b3-a37f378255c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:51.458572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2245ec-bfb3-406b-b6b3-a37f378255c0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:51.458885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2245ec-bfb3-406b-b6b3-a37f378255c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:51.459196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2245ec-bfb3-406b-b6b3-a37f378255c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:51.469723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2245ec-bfb3-406b-b6b3-a37f378255c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-1d2245ec-bfb3-406b-b6b3-a37f378255c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:43:51.477459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:51.478620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:51.479699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 632/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:43:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:51.485428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25bc56-733d-461a-b434-1a61aca06ec6 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:51.584589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25bc56-733d-461a-b434-1a61aca06ec6 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:51.614133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:51.618853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e73d-59d9-4af6-b1a4-e6281b82ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:51.619521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e73d-59d9-4af6-b1a4-e6281b82ab00 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:51.619806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e73d-59d9-4af6-b1a4-e6281b82ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:51.620087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e73d-59d9-4af6-b1a4-e6281b82ab00 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:51.620826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0e73d-59d9-4af6-b1a4-e6281b82ab00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:51.621144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0e73d-59d9-4af6-b1a4-e6281b82ab00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:51.622554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0e73d-59d9-4af6-b1a4-e6281b82ab00 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:51.630401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 618/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:43:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:51.636074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2245ec-bfb3-406b-b6b3-a37f378255c0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:51.718897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:51.742171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2245ec-bfb3-406b-b6b3-a37f378255c0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:51.775999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbc08b-5e7f-4171-8df3-66e9b3b859d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:51.776909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbc08b-5e7f-4171-8df3-66e9b3b859d8 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:51.777284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbc08b-5e7f-4171-8df3-66e9b3b859d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:51.778248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbc08b-5e7f-4171-8df3-66e9b3b859d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:51.788870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cbc08b-5e7f-4171-8df3-66e9b3b859d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:51.790329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cbc08b-5e7f-4171-8df3-66e9b3b859d8 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-51cbc08b-5e7f-4171-8df3-66e9b3b859d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:51.807253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:51.808423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:51.809466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:51.809466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:51.809466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:51.809466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:51.809466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 633/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:51.814384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e73d-59d9-4af6-b1a4-e6281b82ab00 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:51.917292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e73d-59d9-4af6-b1a4-e6281b82ab00 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:51.950222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9756141-c68c-46ae-8a5d-5a848d5e5908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:51.950908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9756141-c68c-46ae-8a5d-5a848d5e5908 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:51.951186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9756141-c68c-46ae-8a5d-5a848d5e5908 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:51.951454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9756141-c68c-46ae-8a5d-5a848d5e5908 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:51.952163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9756141-c68c-46ae-8a5d-5a848d5e5908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:51.952464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9756141-c68c-46ae-8a5d-5a848d5e5908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:51.953817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9756141-c68c-46ae-8a5d-5a848d5e5908 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:51.965546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 619/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:43:51.971238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbc08b-5e7f-4171-8df3-66e9b3b859d8 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:52.076125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbc08b-5e7f-4171-8df3-66e9b3b859d8 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:52.101908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:52.109447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8434d62-04e2-4dd7-bd52-ccc947ac8e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:52.110322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8434d62-04e2-4dd7-bd52-ccc947ac8e52 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:52.110582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8434d62-04e2-4dd7-bd52-ccc947ac8e52 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:52.110857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8434d62-04e2-4dd7-bd52-ccc947ac8e52 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:52.118106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8434d62-04e2-4dd7-bd52-ccc947ac8e52 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:52.119603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8434d62-04e2-4dd7-bd52-ccc947ac8e52 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-e8434d62-04e2-4dd7-bd52-ccc947ac8e52 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:43:52.134075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:52.139719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:52.140941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 634/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:52.143015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9756141-c68c-46ae-8a5d-5a848d5e5908 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:52.200085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:52.243899 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9756141-c68c-46ae-8a5d-5a848d5e5908 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:52.277748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451319ce-ec48-4182-8de5-ec54a6f315e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:52.278414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451319ce-ec48-4182-8de5-ec54a6f315e9 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:52.278727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451319ce-ec48-4182-8de5-ec54a6f315e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:52.279004 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451319ce-ec48-4182-8de5-ec54a6f315e9 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:52.279824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451319ce-ec48-4182-8de5-ec54a6f315e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:52.280132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451319ce-ec48-4182-8de5-ec54a6f315e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:52.281550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451319ce-ec48-4182-8de5-ec54a6f315e9 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:52.298940 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 620/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 05:43:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:52.304735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8434d62-04e2-4dd7-bd52-ccc947ac8e52 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:52.406405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8434d62-04e2-4dd7-bd52-ccc947ac8e52 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:52.438423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e937ae9-1a02-4558-a024-665360cf4ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:52.439090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e937ae9-1a02-4558-a024-665360cf4ad0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:52.439394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e937ae9-1a02-4558-a024-665360cf4ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:52.439667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e937ae9-1a02-4558-a024-665360cf4ad0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:52.493548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:52.505158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e937ae9-1a02-4558-a024-665360cf4ad0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:52.506574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e937ae9-1a02-4558-a024-665360cf4ad0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:52.601571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 635/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:43:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:43:52.607949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451319ce-ec48-4182-8de5-ec54a6f315e9 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:52.727265 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451319ce-ec48-4182-8de5-ec54a6f315e9 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:52.747209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:52.769397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d44ddd-a8e2-47e7-b790-8c307e44dde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:52.770127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d44ddd-a8e2-47e7-b790-8c307e44dde0 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:52.770405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d44ddd-a8e2-47e7-b790-8c307e44dde0 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:52.770705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d44ddd-a8e2-47e7-b790-8c307e44dde0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:52.781016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d44ddd-a8e2-47e7-b790-8c307e44dde0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:52.782439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d44ddd-a8e2-47e7-b790-8c307e44dde0 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-51d44ddd-a8e2-47e7-b790-8c307e44dde0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:52.800039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:52.801112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:52.802247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:52.802247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:52.802247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:52.802247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:52.802247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 621/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:52.806955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e937ae9-1a02-4558-a024-665360cf4ad0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:52.891828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:52.904381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e937ae9-1a02-4558-a024-665360cf4ad0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:52.937939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588b9be-675e-47c3-af65-570c9f33260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:52.938665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588b9be-675e-47c3-af65-570c9f33260d None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:52.938945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588b9be-675e-47c3-af65-570c9f33260d None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:52.939215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588b9be-675e-47c3-af65-570c9f33260d None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:52.939926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588b9be-675e-47c3-af65-570c9f33260d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:52.940224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588b9be-675e-47c3-af65-570c9f33260d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:52.941576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588b9be-675e-47c3-af65-570c9f33260d None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:52.952445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 636/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:52.958234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d44ddd-a8e2-47e7-b790-8c307e44dde0 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:53.058071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d44ddd-a8e2-47e7-b790-8c307e44dde0 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:53.091987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea3b12-7f89-42b2-b9ad-da184690d9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:53.092776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea3b12-7f89-42b2-b9ad-da184690d9ce None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:53.093056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea3b12-7f89-42b2-b9ad-da184690d9ce None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:53.093328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea3b12-7f89-42b2-b9ad-da184690d9ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:53.099852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea3b12-7f89-42b2-b9ad-da184690d9ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:53.101289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea3b12-7f89-42b2-b9ad-da184690d9ce None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-1bea3b12-7f89-42b2-b9ad-da184690d9ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:43:53.114799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:53.115862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:53.116989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:43:53.116989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:43:53.116989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:43:53.116989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:43:53.116989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:43:53.116989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:43:53.116989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:43:53.116989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:53.116989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 622/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:43:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:53.121858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588b9be-675e-47c3-af65-570c9f33260d None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:53.219524 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588b9be-675e-47c3-af65-570c9f33260d None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:53.251678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11407941-7e2f-4987-818a-3e0e73fad2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:53.252319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11407941-7e2f-4987-818a-3e0e73fad2a2 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:53.252596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11407941-7e2f-4987-818a-3e0e73fad2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:53.252993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11407941-7e2f-4987-818a-3e0e73fad2a2 None admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:53.253747 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11407941-7e2f-4987-818a-3e0e73fad2a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:53.254052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11407941-7e2f-4987-818a-3e0e73fad2a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:53.255488 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11407941-7e2f-4987-818a-3e0e73fad2a2 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:53.269935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 637/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:43:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:53.274489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea3b12-7f89-42b2-b9ad-da184690d9ce None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:53.301193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:53.376791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea3b12-7f89-42b2-b9ad-da184690d9ce None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:53.410511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2d390-6dab-4c5b-a3f6-636e631cae52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:53.411172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2d390-6dab-4c5b-a3f6-636e631cae52 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:53.411455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2d390-6dab-4c5b-a3f6-636e631cae52 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:53.411730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2d390-6dab-4c5b-a3f6-636e631cae52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:53.412601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2d390-6dab-4c5b-a3f6-636e631cae52 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=dbfcf2e915c049f08a9e41c555401a6e) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:53.413021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2d390-6dab-4c5b-a3f6-636e631cae52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:53.414452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2d390-6dab-4c5b-a3f6-636e631cae52 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:53.419864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 623/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:43:53 2022] GET /identity/v3/role_assignments?user.id=dbfcf2e915c049f08a9e41c555401a6e&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=7a833c45603a4830b888e58e1232712a => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:55.408598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082864a-b187-48b7-aa17-b90ffa0d56e9 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:55.408598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082864a-b187-48b7-aa17-b90ffa0d56e9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:55.408598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082864a-b187-48b7-aa17-b90ffa0d56e9 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:55.409545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 638/1261] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:55.414909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed70882-5ef6-4df5-afd1-e475bd3a593a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:55.415350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed70882-5ef6-4df5-afd1-e475bd3a593a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:55.415467 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed70882-5ef6-4df5-afd1-e475bd3a593a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:55.437489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-bed70882-5ef6-4df5-afd1-e475bd3a593a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:55.438754 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bed70882-5ef6-4df5-afd1-e475bd3a593a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:55.504804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 624/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:55.527224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082864a-b187-48b7-aa17-b90ffa0d56e9 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:55.538342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:55.629574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082864a-b187-48b7-aa17-b90ffa0d56e9 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:55.662137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d6b00-2933-478f-9344-41dabb434eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:55.662898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d6b00-2933-478f-9344-41dabb434eb5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:55.663165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d6b00-2933-478f-9344-41dabb434eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:55.663437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d6b00-2933-478f-9344-41dabb434eb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:55.674639 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d6b00-2933-478f-9344-41dabb434eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-6f3d6b00-2933-478f-9344-41dabb434eb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:43:55.682167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:55.683333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:55.684301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 639/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:43:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:55.689418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed70882-5ef6-4df5-afd1-e475bd3a593a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:55.707532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:55.786463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed70882-5ef6-4df5-afd1-e475bd3a593a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:55.819382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa3b70-3566-467e-a98a-e0ef6faebd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:55.820052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa3b70-3566-467e-a98a-e0ef6faebd66 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:55.820317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa3b70-3566-467e-a98a-e0ef6faebd66 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:55.820588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa3b70-3566-467e-a98a-e0ef6faebd66 None admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:55.821312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa3b70-3566-467e-a98a-e0ef6faebd66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:55.821620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa3b70-3566-467e-a98a-e0ef6faebd66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:55.823002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa3b70-3566-467e-a98a-e0ef6faebd66 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:55.830839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 625/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:43:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:55.836819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d6b00-2933-478f-9344-41dabb434eb5 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:55.903386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:55.937165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d6b00-2933-478f-9344-41dabb434eb5 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:55.970835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d5efa-9d5b-4b77-805a-8c212a3f7d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:55.971750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d5efa-9d5b-4b77-805a-8c212a3f7d35 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:55.972036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d5efa-9d5b-4b77-805a-8c212a3f7d35 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:55.972358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d5efa-9d5b-4b77-805a-8c212a3f7d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:55.983428 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d5efa-9d5b-4b77-805a-8c212a3f7d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:55.984892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d5efa-9d5b-4b77-805a-8c212a3f7d35 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-bd5d5efa-9d5b-4b77-805a-8c212a3f7d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:56.001454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:56.004388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:56.005511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:56.005511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:56.005511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:56.005511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:56.005511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 640/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:56.008881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa3b70-3566-467e-a98a-e0ef6faebd66 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:56.093017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:56.111056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa3b70-3566-467e-a98a-e0ef6faebd66 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:56.143254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd9e5c-5af6-4da1-85e9-b2cf3a71f58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:56.143907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd9e5c-5af6-4da1-85e9-b2cf3a71f58b None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:56.144185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd9e5c-5af6-4da1-85e9-b2cf3a71f58b None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:56.144458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd9e5c-5af6-4da1-85e9-b2cf3a71f58b None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:56.145161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd9e5c-5af6-4da1-85e9-b2cf3a71f58b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:56.145463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd9e5c-5af6-4da1-85e9-b2cf3a71f58b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:56.146832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd9e5c-5af6-4da1-85e9-b2cf3a71f58b None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:56.158098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 626/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:56.164243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d5efa-9d5b-4b77-805a-8c212a3f7d35 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:56.264887 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d5efa-9d5b-4b77-805a-8c212a3f7d35 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:56.284593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:56.297667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f94e3-ad71-4015-b8d9-5b4418834cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:56.298459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f94e3-ad71-4015-b8d9-5b4418834cf5 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:56.298768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f94e3-ad71-4015-b8d9-5b4418834cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:56.299038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f94e3-ad71-4015-b8d9-5b4418834cf5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:56.306119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f94e3-ad71-4015-b8d9-5b4418834cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:56.307586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f94e3-ad71-4015-b8d9-5b4418834cf5 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-588f94e3-ad71-4015-b8d9-5b4418834cf5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:43:56.321511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:56.325741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:56.326941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 641/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:56.329354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd9e5c-5af6-4da1-85e9-b2cf3a71f58b None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:56.426704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd9e5c-5af6-4da1-85e9-b2cf3a71f58b None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:56.460328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a9b3a-8e25-4b89-8985-36b118bd6494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:56.461002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a9b3a-8e25-4b89-8985-36b118bd6494 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:56.461287 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a9b3a-8e25-4b89-8985-36b118bd6494 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:56.461566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a9b3a-8e25-4b89-8985-36b118bd6494 None admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:56.462396 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a9b3a-8e25-4b89-8985-36b118bd6494 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:56.464143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a9b3a-8e25-4b89-8985-36b118bd6494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:56.464261 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a9b3a-8e25-4b89-8985-36b118bd6494 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:56.473255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:56.481497 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 627/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 05:43:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:56.487322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f94e3-ad71-4015-b8d9-5b4418834cf5 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:56.586085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f94e3-ad71-4015-b8d9-5b4418834cf5 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:56.619840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e06b-6f19-4cba-b730-0a7f8c3037aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:56.620633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e06b-6f19-4cba-b730-0a7f8c3037aa None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:56.620919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e06b-6f19-4cba-b730-0a7f8c3037aa None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:56.621725 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e06b-6f19-4cba-b730-0a7f8c3037aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:56.632353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756e06b-6f19-4cba-b730-0a7f8c3037aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:56.633824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756e06b-6f19-4cba-b730-0a7f8c3037aa None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-3756e06b-6f19-4cba-b730-0a7f8c3037aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:56.651345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:56.652498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:56.653781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:56.653781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:56.653781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:56.653781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:56.653781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 642/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:56.658233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a9b3a-8e25-4b89-8985-36b118bd6494 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:56.761718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a9b3a-8e25-4b89-8985-36b118bd6494 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:56.795696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfcc65-33d1-4587-ac2a-305dc44808d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:56.796373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfcc65-33d1-4587-ac2a-305dc44808d1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:56.796657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfcc65-33d1-4587-ac2a-305dc44808d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:56.796933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfcc65-33d1-4587-ac2a-305dc44808d1 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:56.797642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfcc65-33d1-4587-ac2a-305dc44808d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:56.797949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfcc65-33d1-4587-ac2a-305dc44808d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:56.799349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfcc65-33d1-4587-ac2a-305dc44808d1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:56.811151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 628/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:56.816984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e06b-6f19-4cba-b730-0a7f8c3037aa None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:56.847656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:56.913681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e06b-6f19-4cba-b730-0a7f8c3037aa None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:56.948064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249392a-9e01-400d-8146-61a24bdc1fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:56.948862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249392a-9e01-400d-8146-61a24bdc1fac None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:56.949162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249392a-9e01-400d-8146-61a24bdc1fac None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:56.949448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249392a-9e01-400d-8146-61a24bdc1fac None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:56.955946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249392a-9e01-400d-8146-61a24bdc1fac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:56.957829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249392a-9e01-400d-8146-61a24bdc1fac None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-8249392a-9e01-400d-8146-61a24bdc1fac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:43:56.973183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:56.974162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:56.975365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:43:56.975365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:43:56.975365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:43:56.975365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:43:56.975365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:43:56.975365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:43:56.975365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:43:56.975365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:43:56.975365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 643/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:56.980269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfcc65-33d1-4587-ac2a-305dc44808d1 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:57.040192 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:57.087926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfcc65-33d1-4587-ac2a-305dc44808d1 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:57.122049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d531b-b7e1-4f67-a482-3566b4d8bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:57.122709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d531b-b7e1-4f67-a482-3566b4d8bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:57.122962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d531b-b7e1-4f67-a482-3566b4d8bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:57.123239 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d531b-b7e1-4f67-a482-3566b4d8bdc1 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:57.124023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d531b-b7e1-4f67-a482-3566b4d8bdc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:57.124332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d531b-b7e1-4f67-a482-3566b4d8bdc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:57.128132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d531b-b7e1-4f67-a482-3566b4d8bdc1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:57.137818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 629/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:57.144001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249392a-9e01-400d-8146-61a24bdc1fac None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:57.235206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:57.243380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249392a-9e01-400d-8146-61a24bdc1fac None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:57.275440 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdbbfd-f660-4d0d-8062-42c725e885c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:57.276315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdbbfd-f660-4d0d-8062-42c725e885c4 None admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:57.276592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdbbfd-f660-4d0d-8062-42c725e885c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:57.276863 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdbbfd-f660-4d0d-8062-42c725e885c4 None admin] PATH_INFO: `/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/dbfcf2e915c049f08a9e41c555401a6e/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:57.279314 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdbbfd-f660-4d0d-8062-42c725e885c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=32cbea18a5314504bb39fcd5fdbaa3c3, user_id=dbfcf2e915c049f08a9e41c555401a6e, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:57.280741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdbbfd-f660-4d0d-8062-42c725e885c4 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:57.357427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 644/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Jul 25 05:43:57 2022] PUT /identity/v3/projects/32cbea18a5314504bb39fcd5fdbaa3c3/users/dbfcf2e915c049f08a9e41c555401a6e/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:43:59.345372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb2e1a-f957-443b-9980-29236df6abe8 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:59.345891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb2e1a-f957-443b-9980-29236df6abe8 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:59.346001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb2e1a-f957-443b-9980-29236df6abe8 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:59.347338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 630/1274] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:59.352434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f45f2-a343-43e9-91bb-6a36a4592594 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:59.352812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f45f2-a343-43e9-91bb-6a36a4592594 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:59.352989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f45f2-a343-43e9-91bb-6a36a4592594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:59.369267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-0e5f45f2-a343-43e9-91bb-6a36a4592594 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:59.373687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0e5f45f2-a343-43e9-91bb-6a36a4592594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:59.435144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 645/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:59.458777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb2e1a-f957-443b-9980-29236df6abe8 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:59.562240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb2e1a-f957-443b-9980-29236df6abe8 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:59.581250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:59.597553 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15808488-7ae1-40b9-92bb-f53b7745210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:59.598274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808488-7ae1-40b9-92bb-f53b7745210d None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:59.598583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808488-7ae1-40b9-92bb-f53b7745210d None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:59.598856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808488-7ae1-40b9-92bb-f53b7745210d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:59.608752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15808488-7ae1-40b9-92bb-f53b7745210d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-15808488-7ae1-40b9-92bb-f53b7745210d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:43:59.616439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:59.617559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:59.618640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 631/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Jul 25 05:43:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:59.626815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f45f2-a343-43e9-91bb-6a36a4592594 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:59.703718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:59.720719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f45f2-a343-43e9-91bb-6a36a4592594 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:59.754616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78580f-f101-4a53-938f-30eb1ce554d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:59.755277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78580f-f101-4a53-938f-30eb1ce554d0 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:59.755635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78580f-f101-4a53-938f-30eb1ce554d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:59.755969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78580f-f101-4a53-938f-30eb1ce554d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:59.756722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d78580f-f101-4a53-938f-30eb1ce554d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:59.757023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d78580f-f101-4a53-938f-30eb1ce554d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:59.758389 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d78580f-f101-4a53-938f-30eb1ce554d0 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:59.767030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 646/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Jul 25 05:43:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:59.772398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15808488-7ae1-40b9-92bb-f53b7745210d None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:59.870883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15808488-7ae1-40b9-92bb-f53b7745210d None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:59.903788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280271e-6d27-42a7-8c0b-706d61032492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:59.904633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280271e-6d27-42a7-8c0b-706d61032492 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:59.904884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280271e-6d27-42a7-8c0b-706d61032492 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:59.905157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280271e-6d27-42a7-8c0b-706d61032492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:59.915591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280271e-6d27-42a7-8c0b-706d61032492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:59.920560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280271e-6d27-42a7-8c0b-706d61032492 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-5280271e-6d27-42a7-8c0b-706d61032492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:59.934587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:59.935731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:59.936867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:43:59.936867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:59.936867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:43:59.936867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:43:59.936867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 632/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:59.941498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78580f-f101-4a53-938f-30eb1ce554d0 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.038265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78580f-f101-4a53-938f-30eb1ce554d0 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.071484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e9897-c516-45fa-bc6d-d76433206335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.072139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e9897-c516-45fa-bc6d-d76433206335 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.072421 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e9897-c516-45fa-bc6d-d76433206335 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.072717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e9897-c516-45fa-bc6d-d76433206335 None admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.073444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e9897-c516-45fa-bc6d-d76433206335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:00.073741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e9897-c516-45fa-bc6d-d76433206335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.075160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e9897-c516-45fa-bc6d-d76433206335 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.086691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 647/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:00.092293 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280271e-6d27-42a7-8c0b-706d61032492 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.123405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:00.194396 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280271e-6d27-42a7-8c0b-706d61032492 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.228067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fa1b0-6d42-486f-ba37-cd488ffe3888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.228862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fa1b0-6d42-486f-ba37-cd488ffe3888 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.229139 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fa1b0-6d42-486f-ba37-cd488ffe3888 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.229408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fa1b0-6d42-486f-ba37-cd488ffe3888 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.236858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fa1b0-6d42-486f-ba37-cd488ffe3888 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.238294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fa1b0-6d42-486f-ba37-cd488ffe3888 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-e23fa1b0-6d42-486f-ba37-cd488ffe3888 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:44:00.253499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:00.254543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:44:00.255838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 633/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:00.260582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e9897-c516-45fa-bc6d-d76433206335 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.264699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:00.357590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e9897-c516-45fa-bc6d-d76433206335 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.389984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f8f3b-666e-438f-a9e1-141bfad445d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.390633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f8f3b-666e-438f-a9e1-141bfad445d9 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.390916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f8f3b-666e-438f-a9e1-141bfad445d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.391189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f8f3b-666e-438f-a9e1-141bfad445d9 None admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.392023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f8f3b-666e-438f-a9e1-141bfad445d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:00.392331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f8f3b-666e-438f-a9e1-141bfad445d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.393771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f8f3b-666e-438f-a9e1-141bfad445d9 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.411556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 648/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Jul 25 05:44:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:00.417641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fa1b0-6d42-486f-ba37-cd488ffe3888 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.507181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:00.517097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fa1b0-6d42-486f-ba37-cd488ffe3888 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.549705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60254b0b-5dc8-4bc1-845c-e3717c555200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.550342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254b0b-5dc8-4bc1-845c-e3717c555200 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.550673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254b0b-5dc8-4bc1-845c-e3717c555200 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.550946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254b0b-5dc8-4bc1-845c-e3717c555200 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.616341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60254b0b-5dc8-4bc1-845c-e3717c555200 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.617774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60254b0b-5dc8-4bc1-845c-e3717c555200 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.704726 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:00.716001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 634/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Jul 25 05:44:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:44:00.721779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f8f3b-666e-438f-a9e1-141bfad445d9 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.782405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:00.818899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f8f3b-666e-438f-a9e1-141bfad445d9 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.856351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846b22e-dddd-478d-b878-0329be5fe850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.857145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846b22e-dddd-478d-b878-0329be5fe850 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.857428 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846b22e-dddd-478d-b878-0329be5fe850 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.857700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846b22e-dddd-478d-b878-0329be5fe850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.868095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846b22e-dddd-478d-b878-0329be5fe850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.869527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846b22e-dddd-478d-b878-0329be5fe850 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-7846b22e-dddd-478d-b878-0329be5fe850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:00.887170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:00.888169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:00.889416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:00.889416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:00.889416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:00.889416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:44:00.889416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 649/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:00.894344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60254b0b-5dc8-4bc1-845c-e3717c555200 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.997822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60254b0b-5dc8-4bc1-845c-e3717c555200 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:01.035061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2416d-8e37-4297-bd92-f6253bb2d010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:01.035752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2416d-8e37-4297-bd92-f6253bb2d010 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.036059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2416d-8e37-4297-bd92-f6253bb2d010 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.036309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2416d-8e37-4297-bd92-f6253bb2d010 None admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.037043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2416d-8e37-4297-bd92-f6253bb2d010 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:01.037352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2416d-8e37-4297-bd92-f6253bb2d010 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:01.038771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2416d-8e37-4297-bd92-f6253bb2d010 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:01.056078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 635/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:01.057381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846b22e-dddd-478d-b878-0329be5fe850 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:01.164424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846b22e-dddd-478d-b878-0329be5fe850 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:01.184014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:01.206215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b6670-0e6b-4408-ad55-2533c2f17b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:01.207080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b6670-0e6b-4408-ad55-2533c2f17b83 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.207418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b6670-0e6b-4408-ad55-2533c2f17b83 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.207727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b6670-0e6b-4408-ad55-2533c2f17b83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.215317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b6670-0e6b-4408-ad55-2533c2f17b83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:01.216985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b6670-0e6b-4408-ad55-2533c2f17b83 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application [None req-a88b6670-0e6b-4408-ad55-2533c2f17b83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:44:01.233043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:01.235044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:01.236180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:44:01.236180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:44:01.236180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:44:01.236180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:44:01.236180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:44:01.236180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:44:01.236180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:01.236180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: ERROR keystone.server.flask.application Jul 25 05:44:01.236180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 650/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Jul 25 05:44:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:01.240276 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2416d-8e37-4297-bd92-f6253bb2d010 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:01.291520 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:01.342450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2416d-8e37-4297-bd92-f6253bb2d010 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:01.377654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23757af7-95fe-4cd7-88a4-aee9beb67968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:01.378345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23757af7-95fe-4cd7-88a4-aee9beb67968 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.378674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23757af7-95fe-4cd7-88a4-aee9beb67968 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.378957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23757af7-95fe-4cd7-88a4-aee9beb67968 None admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.379733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23757af7-95fe-4cd7-88a4-aee9beb67968 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:01.380040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23757af7-95fe-4cd7-88a4-aee9beb67968 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:01.381467 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23757af7-95fe-4cd7-88a4-aee9beb67968 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:01.394564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 636/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Jul 25 05:44:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:01.400709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b6670-0e6b-4408-ad55-2533c2f17b83 None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:01.502153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b6670-0e6b-4408-ad55-2533c2f17b83 None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:01.535611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54504088-fa16-45ce-9869-14ecc9884402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:01.536252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54504088-fa16-45ce-9869-14ecc9884402 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.536529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54504088-fa16-45ce-9869-14ecc9884402 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.536805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54504088-fa16-45ce-9869-14ecc9884402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.537623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54504088-fa16-45ce-9869-14ecc9884402 None admin] RBAC: Adding query filter params (role.id=7a833c45603a4830b888e58e1232712a, scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3, user.id=dbfcf2e915c049f08a9e41c555401a6e) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:01.537930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54504088-fa16-45ce-9869-14ecc9884402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:01.539436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54504088-fa16-45ce-9869-14ecc9884402 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:01.545133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 651/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Jul 25 05:44:01 2022] GET /identity/v3/role_assignments?user.id=dbfcf2e915c049f08a9e41c555401a6e&scope.project.id=32cbea18a5314504bb39fcd5fdbaa3c3&role.id=7a833c45603a4830b888e58e1232712a => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:03.567052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329b0da-cfbe-4fb7-a348-1a872142117a None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:03.567616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329b0da-cfbe-4fb7-a348-1a872142117a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:03.567616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329b0da-cfbe-4fb7-a348-1a872142117a None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:03.569019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 637/1288] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:03.574993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ec2af-33c4-423c-9cdf-5bce50650141 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:03.575495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ec2af-33c4-423c-9cdf-5bce50650141 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:03.575969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ec2af-33c4-423c-9cdf-5bce50650141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:03.597768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-3f3ec2af-33c4-423c-9cdf-5bce50650141 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:03.599006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3f3ec2af-33c4-423c-9cdf-5bce50650141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:03.627232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:03.663853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 652/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:03.686179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329b0da-cfbe-4fb7-a348-1a872142117a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:03.786752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329b0da-cfbe-4fb7-a348-1a872142117a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:03.797345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:03.819657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3b29c-84cc-4c05-82ce-85cdc420dcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:03.820377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3b29c-84cc-4c05-82ce-85cdc420dcb8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:03.820652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3b29c-84cc-4c05-82ce-85cdc420dcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:03.820919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3b29c-84cc-4c05-82ce-85cdc420dcb8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:03.821597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3b29c-84cc-4c05-82ce-85cdc420dcb8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:03.823008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3b29c-84cc-4c05-82ce-85cdc420dcb8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-3ed3b29c-84cc-4c05-82ce-85cdc420dcb8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:03.835210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:03.836523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:44:03.837653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 638/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 05:44:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:03.843307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ec2af-33c4-423c-9cdf-5bce50650141 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:03.940447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ec2af-33c4-423c-9cdf-5bce50650141 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:03.974328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d4a62-f1f3-4c56-9f45-4bfbe6556597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:03.975011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d4a62-f1f3-4c56-9f45-4bfbe6556597 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:03.975290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d4a62-f1f3-4c56-9f45-4bfbe6556597 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:03.975564 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d4a62-f1f3-4c56-9f45-4bfbe6556597 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:03.976261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d4a62-f1f3-4c56-9f45-4bfbe6556597 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:03.976558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d4a62-f1f3-4c56-9f45-4bfbe6556597 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:03.977925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d4a62-f1f3-4c56-9f45-4bfbe6556597 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:03.982659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 653/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Jul 25 05:44:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:03.988446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3b29c-84cc-4c05-82ce-85cdc420dcb8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:04.091314 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3b29c-84cc-4c05-82ce-85cdc420dcb8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:04.123806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a420a07-938f-43ac-b054-0bfdb1c02520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:04.124458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a420a07-938f-43ac-b054-0bfdb1c02520 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:04.124736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a420a07-938f-43ac-b054-0bfdb1c02520 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:04.125007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a420a07-938f-43ac-b054-0bfdb1c02520 None admin] PATH_INFO: `/v3/services` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:04.125693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a420a07-938f-43ac-b054-0bfdb1c02520 None admin] RBAC: Adding query filter params (type=placement) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:04.125993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a420a07-938f-43ac-b054-0bfdb1c02520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:04.127365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a420a07-938f-43ac-b054-0bfdb1c02520 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:04.131710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 639/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Jul 25 05:44:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:06.143644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea484053-b51e-4eb2-af76-a32bcb1d152d None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:06.144168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea484053-b51e-4eb2-af76-a32bcb1d152d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:06.144168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea484053-b51e-4eb2-af76-a32bcb1d152d None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:06.145581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 654/1293] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:06.150797 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829416af-0fb7-4caa-abfd-dd05ca330688 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:06.151224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829416af-0fb7-4caa-abfd-dd05ca330688 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:06.151341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829416af-0fb7-4caa-abfd-dd05ca330688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:06.167886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-829416af-0fb7-4caa-abfd-dd05ca330688 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:06.169009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-829416af-0fb7-4caa-abfd-dd05ca330688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:06.198205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:06.233662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 640/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:06.256669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea484053-b51e-4eb2-af76-a32bcb1d152d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:06.287756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:06.355654 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea484053-b51e-4eb2-af76-a32bcb1d152d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:06.395343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f3631-4901-4f65-8117-ce299a3d3b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:06.396008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f3631-4901-4f65-8117-ce299a3d3b39 None admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:06.396288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f3631-4901-4f65-8117-ce299a3d3b39 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:06.396593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f3631-4901-4f65-8117-ce299a3d3b39 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:06.397326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f3631-4901-4f65-8117-ce299a3d3b39 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:06.398720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f3631-4901-4f65-8117-ce299a3d3b39 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:06.488455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 655/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Jul 25 05:44:06 2022] POST /identity/v3/services => generated 246 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:08.538061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea33e59-898b-4d25-ac22-de348ec48b2b None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:08.538633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea33e59-898b-4d25-ac22-de348ec48b2b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:08.538633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea33e59-898b-4d25-ac22-de348ec48b2b None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:08.540150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 641/1296] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:08.545705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf857b-5bd2-43ab-a669-4ec9a44b4b9e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:08.546103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf857b-5bd2-43ab-a669-4ec9a44b4b9e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:08.546261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf857b-5bd2-43ab-a669-4ec9a44b4b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:08.563127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-05cf857b-5bd2-43ab-a669-4ec9a44b4b9e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:08.564318 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-05cf857b-5bd2-43ab-a669-4ec9a44b4b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:08.638974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 656/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:08.662053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea33e59-898b-4d25-ac22-de348ec48b2b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:08.773502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea33e59-898b-4d25-ac22-de348ec48b2b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:08.809685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df4fc8-4d96-459b-833d-4a7ab9e1a4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:08.810407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df4fc8-4d96-459b-833d-4a7ab9e1a4a8 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:08.810720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df4fc8-4d96-459b-833d-4a7ab9e1a4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:08.810993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df4fc8-4d96-459b-833d-4a7ab9e1a4a8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:08.811640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df4fc8-4d96-459b-833d-4a7ab9e1a4a8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:08.812991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df4fc8-4d96-459b-833d-4a7ab9e1a4a8 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-b6df4fc8-4d96-459b-833d-4a7ab9e1a4a8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:08.825213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:08.828611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:44:08.829724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 642/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 05:44:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:08.832454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf857b-5bd2-43ab-a669-4ec9a44b4b9e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:08.852088 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:08.930118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf857b-5bd2-43ab-a669-4ec9a44b4b9e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:08.965437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486533a-15fc-4d1c-aa30-770cbbd7cd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:08.966099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486533a-15fc-4d1c-aa30-770cbbd7cd2b None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:08.966380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486533a-15fc-4d1c-aa30-770cbbd7cd2b None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:08.966685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486533a-15fc-4d1c-aa30-770cbbd7cd2b None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:08.967396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486533a-15fc-4d1c-aa30-770cbbd7cd2b None admin] RBAC: Adding query filter params (name=placement) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:08.967706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486533a-15fc-4d1c-aa30-770cbbd7cd2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:08.969127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486533a-15fc-4d1c-aa30-770cbbd7cd2b None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:08.974260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 657/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Jul 25 05:44:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:08.980252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df4fc8-4d96-459b-833d-4a7ab9e1a4a8 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:09.010748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:09.079149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df4fc8-4d96-459b-833d-4a7ab9e1a4a8 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:09.112572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713500a-17af-47cc-b108-ecb943798d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:09.113224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713500a-17af-47cc-b108-ecb943798d92 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:09.113507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713500a-17af-47cc-b108-ecb943798d92 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:09.113784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713500a-17af-47cc-b108-ecb943798d92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:09.114593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713500a-17af-47cc-b108-ecb943798d92 None admin] RBAC: Adding query filter params (interface=public, service_id=b8b68df9c29e468d85a087541a03c84e, region_id=RegionOne) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:09.114904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713500a-17af-47cc-b108-ecb943798d92 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:09.116266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713500a-17af-47cc-b108-ecb943798d92 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:09.121753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 643/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Jul 25 05:44:08 2022] GET /identity/v3/endpoints?service_id=b8b68df9c29e468d85a087541a03c84e&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:09.127457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486533a-15fc-4d1c-aa30-770cbbd7cd2b None admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:09.206395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:09.223186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486533a-15fc-4d1c-aa30-770cbbd7cd2b None admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:09.255253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ccac1-e1f1-4296-bc49-efcadddfa610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:09.255905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ccac1-e1f1-4296-bc49-efcadddfa610 None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:09.256186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ccac1-e1f1-4296-bc49-efcadddfa610 None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:09.256460 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ccac1-e1f1-4296-bc49-efcadddfa610 None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:09.257137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ccac1-e1f1-4296-bc49-efcadddfa610 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:09.258609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ccac1-e1f1-4296-bc49-efcadddfa610 None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:09.266099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 658/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Jul 25 05:44:09 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:11.291503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ce1b-0977-463e-94d8-8a2481732094 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:11.292011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ce1b-0977-463e-94d8-8a2481732094 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:11.292011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ce1b-0977-463e-94d8-8a2481732094 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:11.293868 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 644/1302] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:11.299232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439a2c4-b36f-423b-a298-9159f4d85f77 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:11.299660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439a2c4-b36f-423b-a298-9159f4d85f77 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:11.299822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439a2c4-b36f-423b-a298-9159f4d85f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:11.316787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e439a2c4-b36f-423b-a298-9159f4d85f77 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:11.322157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e439a2c4-b36f-423b-a298-9159f4d85f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:11.382342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 659/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:11.405908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193ce1b-0977-463e-94d8-8a2481732094 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:11.498415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:11.504925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193ce1b-0977-463e-94d8-8a2481732094 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:11.538701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a79ce-f4cf-401e-9f1b-a0d1077d3960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:11.539487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a79ce-f4cf-401e-9f1b-a0d1077d3960 None admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:11.539765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a79ce-f4cf-401e-9f1b-a0d1077d3960 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:11.540035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a79ce-f4cf-401e-9f1b-a0d1077d3960 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:11.540673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a79ce-f4cf-401e-9f1b-a0d1077d3960 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:11.542071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a79ce-f4cf-401e-9f1b-a0d1077d3960 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application [None req-171a79ce-f4cf-401e-9f1b-a0d1077d3960 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:11.554290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:11.555554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: ERROR keystone.server.flask.application Jul 25 05:44:11.556641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 645/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Jul 25 05:44:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:11.561658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439a2c4-b36f-423b-a298-9159f4d85f77 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:11.637135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:11.659690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439a2c4-b36f-423b-a298-9159f4d85f77 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:11.693690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475e69b-909c-498c-a624-d200b665aa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:11.694372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475e69b-909c-498c-a624-d200b665aa5c None admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:11.694684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475e69b-909c-498c-a624-d200b665aa5c None admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:11.694961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475e69b-909c-498c-a624-d200b665aa5c None admin] PATH_INFO: `/v3/services` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:11.695661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475e69b-909c-498c-a624-d200b665aa5c None admin] RBAC: Adding query filter params (name=placement) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:11.695967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475e69b-909c-498c-a624-d200b665aa5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:11.697334 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475e69b-909c-498c-a624-d200b665aa5c None admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:11.702394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 660/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Jul 25 05:44:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:11.708489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a79ce-f4cf-401e-9f1b-a0d1077d3960 None admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:11.807483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a79ce-f4cf-401e-9f1b-a0d1077d3960 None admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:11.841281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fd5d6-bd4f-4fb9-8bb0-e07b56aa80a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:11.841938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fd5d6-bd4f-4fb9-8bb0-e07b56aa80a1 None admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:11.842219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fd5d6-bd4f-4fb9-8bb0-e07b56aa80a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:11.842515 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fd5d6-bd4f-4fb9-8bb0-e07b56aa80a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:11.843247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807fd5d6-bd4f-4fb9-8bb0-e07b56aa80a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:11.844613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807fd5d6-bd4f-4fb9-8bb0-e07b56aa80a1 None admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:11.916909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 646/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Jul 25 05:44:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:21.699405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=30 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:21.912153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:07.676901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aacd89-5392-4c12-a37f-107f9dcec0f6 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:07.677762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aacd89-5392-4c12-a37f-107f9dcec0f6 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:07.678224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aacd89-5392-4c12-a37f-107f9dcec0f6 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:07.685501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe6c84-f4ae-4c9d-a908-c54b35db24d4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:07.686306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe6c84-f4ae-4c9d-a908-c54b35db24d4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:07.686832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe6c84-f4ae-4c9d-a908-c54b35db24d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:07.690459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 661/1307] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:07 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:07.781114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-9efe6c84-f4ae-4c9d-a908-c54b35db24d4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:07.782602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9efe6c84-f4ae-4c9d-a908-c54b35db24d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:07.951732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 647/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:50:07.986982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f975a00-123a-4601-beb1-aa553e70098f None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:07.986982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f975a00-123a-4601-beb1-aa553e70098f None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:07.986982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f975a00-123a-4601-beb1-aa553e70098f None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:07.988605 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 662/1309] 10.222.0.17 () {54 vars in 965 bytes} [Mon Jul 25 05:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:08.004692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da61e48-1d9f-4a6d-9286-04db651b2dff None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:08.005216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da61e48-1d9f-4a6d-9286-04db651b2dff None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:08.005706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da61e48-1d9f-4a6d-9286-04db651b2dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:08.159819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3da61e48-1d9f-4a6d-9286-04db651b2dff None None] MFA Rules not processed for user `ecd61347f8874b89aada19772ec56179`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:08.161449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3da61e48-1d9f-4a6d-9286-04db651b2dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:08.193201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:08.260552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 648/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Mon Jul 25 05:50:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:50:08.266166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f975a00-123a-4601-beb1-aa553e70098f None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:08.268115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1f975a00-123a-4601-beb1-aa553e70098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:08.381222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f975a00-123a-4601-beb1-aa553e70098f None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:08.461352 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b9316-0df7-4539-9dd0-514a8f4cc161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:08.461920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b9316-0df7-4539-9dd0-514a8f4cc161 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:08.462198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b9316-0df7-4539-9dd0-514a8f4cc161 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:08.462502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b9316-0df7-4539-9dd0-514a8f4cc161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:08.464122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d37b9316-0df7-4539-9dd0-514a8f4cc161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:08.520366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:08.583701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b9316-0df7-4539-9dd0-514a8f4cc161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:08.585185 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b9316-0df7-4539-9dd0-514a8f4cc161 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:08.608501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 663/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:50:10.963253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c41aa-37b8-48d8-95c0-c5c131bdd905 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:10.963816 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c41aa-37b8-48d8-95c0-c5c131bdd905 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:10.963816 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c41aa-37b8-48d8-95c0-c5c131bdd905 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:10.965496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 649/1312] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:10.971138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7f567-1055-4ba7-b38d-1a71c871510b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:10.971631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7f567-1055-4ba7-b38d-1a71c871510b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:10.971710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7f567-1055-4ba7-b38d-1a71c871510b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:11.048352 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-64c7f567-1055-4ba7-b38d-1a71c871510b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:11.049565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-64c7f567-1055-4ba7-b38d-1a71c871510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:11.149046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 664/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:11.205268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb01d63-bc8c-4389-98c1-1660f2180fe4 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:11.205860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb01d63-bc8c-4389-98c1-1660f2180fe4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:11.206216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb01d63-bc8c-4389-98c1-1660f2180fe4 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:11.207798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 650/1314] 10.222.0.17 () {54 vars in 965 bytes} [Mon Jul 25 05:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:11.213970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e295901-6be3-4898-9167-c53b478e3d24 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:11.214354 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e295901-6be3-4898-9167-c53b478e3d24 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:11.215376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e295901-6be3-4898-9167-c53b478e3d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:11.226537 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:11.298011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-6e295901-6be3-4898-9167-c53b478e3d24 None None] MFA Rules not processed for user `ecd61347f8874b89aada19772ec56179`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:11.299324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6e295901-6be3-4898-9167-c53b478e3d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:11.376111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 665/1315] 10.222.0.17 () {58 vars in 1062 bytes} [Mon Jul 25 05:50:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:11.384929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb01d63-bc8c-4389-98c1-1660f2180fe4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:11.386720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7cb01d63-bc8c-4389-98c1-1660f2180fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:11.490555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb01d63-bc8c-4389-98c1-1660f2180fe4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:11.562037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:11.564710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f181e9-b0bc-472c-83bd-897d3c06aa8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:11.565361 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f181e9-b0bc-472c-83bd-897d3c06aa8e service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:11.565641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f181e9-b0bc-472c-83bd-897d3c06aa8e service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:11.565912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f181e9-b0bc-472c-83bd-897d3c06aa8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:11.567603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a9f181e9-b0bc-472c-83bd-897d3c06aa8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:11.675050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f181e9-b0bc-472c-83bd-897d3c06aa8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:11.676484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f181e9-b0bc-472c-83bd-897d3c06aa8e service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:11.695425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 651/1316] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:14.460352 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf594c1-e129-4236-be66-042391decd08 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:14.460932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf594c1-e129-4236-be66-042391decd08 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:14.460932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf594c1-e129-4236-be66-042391decd08 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:14.462296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 666/1317] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:14.468110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edd045-00d3-4c51-bcc7-a0f152dd4e9e None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:14.468536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edd045-00d3-4c51-bcc7-a0f152dd4e9e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:14.468676 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edd045-00d3-4c51-bcc7-a0f152dd4e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:14.531874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-80edd045-00d3-4c51-bcc7-a0f152dd4e9e None None] MFA Rules not processed for user `73a2be1f4ead4bf0a051368674237133`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:14.533097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-80edd045-00d3-4c51-bcc7-a0f152dd4e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:14.584653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:14.632351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 652/1318] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:14.672983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf594c1-e129-4236-be66-042391decd08 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:14.674550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ebf594c1-e129-4236-be66-042391decd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:14.774149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf594c1-e129-4236-be66-042391decd08 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:14.776853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:14.845719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb91758-d189-412b-be2a-60afda8784ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:14.846387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb91758-d189-412b-be2a-60afda8784ca service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:14.846696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb91758-d189-412b-be2a-60afda8784ca service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:14.846970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb91758-d189-412b-be2a-60afda8784ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:14.848600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2bb91758-d189-412b-be2a-60afda8784ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:14.955574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb91758-d189-412b-be2a-60afda8784ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:14.957023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb91758-d189-412b-be2a-60afda8784ca service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:14.975474 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 667/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:17.777614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df991b3c-85a4-4bdd-a1df-107a4c9080ba None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:17.778177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df991b3c-85a4-4bdd-a1df-107a4c9080ba None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:17.778177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df991b3c-85a4-4bdd-a1df-107a4c9080ba None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:17.779862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 653/1320] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:17.785635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963e9d2-5cdb-4fd5-8c33-6cf8be36073e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:17.786063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963e9d2-5cdb-4fd5-8c33-6cf8be36073e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:17.786213 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963e9d2-5cdb-4fd5-8c33-6cf8be36073e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:17.825212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:17.849468 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-0963e9d2-5cdb-4fd5-8c33-6cf8be36073e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:17.850780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0963e9d2-5cdb-4fd5-8c33-6cf8be36073e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:17.943603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 668/1321] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:17.982810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df991b3c-85a4-4bdd-a1df-107a4c9080ba None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:17.984375 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-df991b3c-85a4-4bdd-a1df-107a4c9080ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:18.088625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df991b3c-85a4-4bdd-a1df-107a4c9080ba None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:18.169003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7e83e-4462-44c4-bbd1-c2b11ede0ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:18.169651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7e83e-4462-44c4-bbd1-c2b11ede0ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:18.169934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7e83e-4462-44c4-bbd1-c2b11ede0ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:18.170214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7e83e-4462-44c4-bbd1-c2b11ede0ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:18.171881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8fa7e83e-4462-44c4-bbd1-c2b11ede0ac0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:18.179201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:18.277737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7e83e-4462-44c4-bbd1-c2b11ede0ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:18.279202 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7e83e-4462-44c4-bbd1-c2b11ede0ac0 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:18.296847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 654/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:21.304734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3342c0-c8d1-4528-8379-ea294274f655 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:21.305230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3342c0-c8d1-4528-8379-ea294274f655 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:21.305323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3342c0-c8d1-4528-8379-ea294274f655 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:21.306704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 669/1323] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:21.312501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b80878-6b64-4327-8674-34dd7da34a28 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:21.312924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b80878-6b64-4327-8674-34dd7da34a28 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:21.313046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b80878-6b64-4327-8674-34dd7da34a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:21.376512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-d5b80878-6b64-4327-8674-34dd7da34a28 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:21.377703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d5b80878-6b64-4327-8674-34dd7da34a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:21.434657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:21.481249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 655/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:21.517972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3342c0-c8d1-4528-8379-ea294274f655 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:21.519530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-db3342c0-c8d1-4528-8379-ea294274f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:21.630246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3342c0-c8d1-4528-8379-ea294274f655 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:21.643609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:21.708818 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee96cc0-99db-4e74-8f04-b16842533784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:21.709473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee96cc0-99db-4e74-8f04-b16842533784 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:21.709750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee96cc0-99db-4e74-8f04-b16842533784 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:21.710021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee96cc0-99db-4e74-8f04-b16842533784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:21.711694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-fee96cc0-99db-4e74-8f04-b16842533784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:21.819404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee96cc0-99db-4e74-8f04-b16842533784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:21.820762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee96cc0-99db-4e74-8f04-b16842533784 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:21.838553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 670/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:23.908906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e8790-0949-4be9-8e23-caacfc0150f4 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:23.908906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e8790-0949-4be9-8e23-caacfc0150f4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:23.908906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e8790-0949-4be9-8e23-caacfc0150f4 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:23.910269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 656/1326] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:23.915607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e08ee-a066-4549-b7ab-9fa80171fecb None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:23.916051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e08ee-a066-4549-b7ab-9fa80171fecb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:23.916378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e08ee-a066-4549-b7ab-9fa80171fecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:23.963086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:23.981841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-b68e08ee-a066-4549-b7ab-9fa80171fecb None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:23.983103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b68e08ee-a066-4549-b7ab-9fa80171fecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:24.082225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 671/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:24.117455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e8790-0949-4be9-8e23-caacfc0150f4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:24.119026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0c6e8790-0949-4be9-8e23-caacfc0150f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:24.221545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e8790-0949-4be9-8e23-caacfc0150f4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:24.294105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f06e69-efe7-4574-94b0-096a027c067e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:24.294790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06e69-efe7-4574-94b0-096a027c067e service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:24.295783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06e69-efe7-4574-94b0-096a027c067e service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:24.296074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f06e69-efe7-4574-94b0-096a027c067e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:24.297796 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-38f06e69-efe7-4574-94b0-096a027c067e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:24.314937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:24.411154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f06e69-efe7-4574-94b0-096a027c067e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:24.412616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f06e69-efe7-4574-94b0-096a027c067e service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:24.432678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 657/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:26.519669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e2eb4-a7d1-4009-a64e-ecaacaeb4db6 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:26.520223 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e2eb4-a7d1-4009-a64e-ecaacaeb4db6 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:26.520223 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e2eb4-a7d1-4009-a64e-ecaacaeb4db6 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:26.521638 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 672/1329] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:26.527499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e9de1-ff72-4d67-9734-1d8118f1b5d2 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:26.528021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e9de1-ff72-4d67-9734-1d8118f1b5d2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:26.528105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e9de1-ff72-4d67-9734-1d8118f1b5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:26.592332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e20e9de1-ff72-4d67-9734-1d8118f1b5d2 None None] MFA Rules not processed for user `73a2be1f4ead4bf0a051368674237133`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:26.593541 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e20e9de1-ff72-4d67-9734-1d8118f1b5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:26.656330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:26.688934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 658/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:26.737005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e2eb4-a7d1-4009-a64e-ecaacaeb4db6 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:26.738552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-945e2eb4-a7d1-4009-a64e-ecaacaeb4db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:26.847372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e2eb4-a7d1-4009-a64e-ecaacaeb4db6 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:26.866829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:26.921736 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdc604-29bd-4c24-9345-0f76e05d6111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:26.922387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc604-29bd-4c24-9345-0f76e05d6111 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:26.922700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc604-29bd-4c24-9345-0f76e05d6111 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:26.923105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc604-29bd-4c24-9345-0f76e05d6111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:26.924743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c9fdc604-29bd-4c24-9345-0f76e05d6111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:27.035399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdc604-29bd-4c24-9345-0f76e05d6111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:27.036853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdc604-29bd-4c24-9345-0f76e05d6111 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:27.054459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 673/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:26 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:29.548861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29f082-8ccf-4f00-8aa4-4be4d403f5c3 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:29.549417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29f082-8ccf-4f00-8aa4-4be4d403f5c3 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:29.549417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29f082-8ccf-4f00-8aa4-4be4d403f5c3 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:29.550819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 659/1332] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:29.556122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa7274-55d4-41bd-b30a-c141c27097ab None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:29.556552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa7274-55d4-41bd-b30a-c141c27097ab None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:29.556663 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa7274-55d4-41bd-b30a-c141c27097ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:29.619230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:29.622139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-0baa7274-55d4-41bd-b30a-c141c27097ab None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:29.623380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0baa7274-55d4-41bd-b30a-c141c27097ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:29.719921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 674/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:29.755133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29f082-8ccf-4f00-8aa4-4be4d403f5c3 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:29.756687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-eb29f082-8ccf-4f00-8aa4-4be4d403f5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:29.857857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29f082-8ccf-4f00-8aa4-4be4d403f5c3 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:29.929630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3657c-51ee-4131-9b02-920ca3249f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:29.930281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3657c-51ee-4131-9b02-920ca3249f36 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:29.930601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3657c-51ee-4131-9b02-920ca3249f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:29.930883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3657c-51ee-4131-9b02-920ca3249f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:29.932523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c0e3657c-51ee-4131-9b02-920ca3249f36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:29.959507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:30.040569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3657c-51ee-4131-9b02-920ca3249f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:30.042002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3657c-51ee-4131-9b02-920ca3249f36 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:30.060045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 660/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:32.120578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d5c3-0704-486d-afac-b8f4a3aefe39 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:32.121156 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d5c3-0704-486d-afac-b8f4a3aefe39 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:32.121156 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59d5c3-0704-486d-afac-b8f4a3aefe39 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:32.122524 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 675/1335] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:32.128286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652f439-bba0-44f0-ad3d-ff3b0f4ac6a2 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:32.128708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652f439-bba0-44f0-ad3d-ff3b0f4ac6a2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:32.128837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652f439-bba0-44f0-ad3d-ff3b0f4ac6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:32.194092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0652f439-bba0-44f0-ad3d-ff3b0f4ac6a2 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:32.195258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0652f439-bba0-44f0-ad3d-ff3b0f4ac6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:32.268072 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:32.290426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 661/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:32.330684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59d5c3-0704-486d-afac-b8f4a3aefe39 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:32.332252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6c59d5c3-0704-486d-afac-b8f4a3aefe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:32.432486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59d5c3-0704-486d-afac-b8f4a3aefe39 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:32.459220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:32.503824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee978d7-3479-4182-82c1-8672c1790fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:32.504465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee978d7-3479-4182-82c1-8672c1790fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:32.504745 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee978d7-3479-4182-82c1-8672c1790fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:32.505038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee978d7-3479-4182-82c1-8672c1790fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:32.506693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-fee978d7-3479-4182-82c1-8672c1790fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:32.614646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee978d7-3479-4182-82c1-8672c1790fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:32.616135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee978d7-3479-4182-82c1-8672c1790fd1 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:32.634498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 676/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:35.519385 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c285f-77ea-4efc-a1c5-e88658d3d1e0 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:35.519947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c285f-77ea-4efc-a1c5-e88658d3d1e0 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:35.519947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c285f-77ea-4efc-a1c5-e88658d3d1e0 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:35.521330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 662/1338] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:35.526589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9b128-798c-481e-baa8-e79f658e9294 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:35.526973 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9b128-798c-481e-baa8-e79f658e9294 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:35.527072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9b128-798c-481e-baa8-e79f658e9294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:35.613395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-3bd9b128-798c-481e-baa8-e79f658e9294 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:35.614644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3bd9b128-798c-481e-baa8-e79f658e9294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:35.623212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:35.712877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 677/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:35.749326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c285f-77ea-4efc-a1c5-e88658d3d1e0 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:35.750870 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-935c285f-77ea-4efc-a1c5-e88658d3d1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:35.851760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c285f-77ea-4efc-a1c5-e88658d3d1e0 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:35.923412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7674d88-d896-439c-b7c5-2121c5458b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:35.924264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7674d88-d896-439c-b7c5-2121c5458b0e service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:35.924545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7674d88-d896-439c-b7c5-2121c5458b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:35.924820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7674d88-d896-439c-b7c5-2121c5458b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:35.927916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e7674d88-d896-439c-b7c5-2121c5458b0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:35.964988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:36.036309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7674d88-d896-439c-b7c5-2121c5458b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:36.037731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7674d88-d896-439c-b7c5-2121c5458b0e service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:36.056653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 663/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:40.357280 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579951b-3a41-4c48-b291-bc1a8af3375a None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:40.357799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579951b-3a41-4c48-b291-bc1a8af3375a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:40.357893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579951b-3a41-4c48-b291-bc1a8af3375a None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:40.359291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 678/1341] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:40.364719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ccae4-5de4-43de-beee-cde7fc51e281 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:40.365143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ccae4-5de4-43de-beee-cde7fc51e281 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:40.366261 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ccae4-5de4-43de-beee-cde7fc51e281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:40.433575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a54ccae4-5de4-43de-beee-cde7fc51e281 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:40.434816 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a54ccae4-5de4-43de-beee-cde7fc51e281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:40.520110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:40.531504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 664/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:40.570790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579951b-3a41-4c48-b291-bc1a8af3375a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:40.572336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5579951b-3a41-4c48-b291-bc1a8af3375a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:40.675539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579951b-3a41-4c48-b291-bc1a8af3375a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:40.713640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:40.750778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67149dc2-a905-4270-ad29-2ff77328560c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:40.751434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67149dc2-a905-4270-ad29-2ff77328560c service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:40.751814 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67149dc2-a905-4270-ad29-2ff77328560c service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:40.752093 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67149dc2-a905-4270-ad29-2ff77328560c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:40.753721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-67149dc2-a905-4270-ad29-2ff77328560c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:40.865775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67149dc2-a905-4270-ad29-2ff77328560c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:40.867271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67149dc2-a905-4270-ad29-2ff77328560c service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:40.886407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 679/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:43.911736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1167d00-2ff8-402c-82a3-7fecfd6a6dfc None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:43.912139 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1167d00-2ff8-402c-82a3-7fecfd6a6dfc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:43.912285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1167d00-2ff8-402c-82a3-7fecfd6a6dfc None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:43.913682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 665/1344] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:43.918945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c1493-adc6-4325-96f1-62a36d83d9be None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:43.919359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c1493-adc6-4325-96f1-62a36d83d9be None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:43.919534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c1493-adc6-4325-96f1-62a36d83d9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:43.982828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-251c1493-adc6-4325-96f1-62a36d83d9be None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:43.983923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-251c1493-adc6-4325-96f1-62a36d83d9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:44.001311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:44.080236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 680/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:44.117122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1167d00-2ff8-402c-82a3-7fecfd6a6dfc None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:44.118800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e1167d00-2ff8-402c-82a3-7fecfd6a6dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:44.221929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1167d00-2ff8-402c-82a3-7fecfd6a6dfc None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:44.296761 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f575f3c-fd7f-470e-9f9f-45bac77560e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:44.297333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575f3c-fd7f-470e-9f9f-45bac77560e8 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:44.297618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575f3c-fd7f-470e-9f9f-45bac77560e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:44.297895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575f3c-fd7f-470e-9f9f-45bac77560e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:44.299557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0f575f3c-fd7f-470e-9f9f-45bac77560e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:44.343466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:44.414127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f575f3c-fd7f-470e-9f9f-45bac77560e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:44.415604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f575f3c-fd7f-470e-9f9f-45bac77560e8 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:44.434640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 666/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:48.929410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8efae-ec37-43e5-8740-eb0daaaac68c None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:48.929919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8efae-ec37-43e5-8740-eb0daaaac68c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:48.930031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8efae-ec37-43e5-8740-eb0daaaac68c None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:48.931398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 681/1347] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:48.936772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628a689-622f-4d1c-a790-7615cfbca327 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:48.937267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628a689-622f-4d1c-a790-7615cfbca327 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:48.937362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628a689-622f-4d1c-a790-7615cfbca327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:49.003519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0628a689-622f-4d1c-a790-7615cfbca327 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:49.004735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0628a689-622f-4d1c-a790-7615cfbca327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:49.105347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:49.107917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 667/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:49.146106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8efae-ec37-43e5-8740-eb0daaaac68c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:49.147769 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e9d8efae-ec37-43e5-8740-eb0daaaac68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:49.253654 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8efae-ec37-43e5-8740-eb0daaaac68c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:49.303367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:49.331526 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fda55-10ee-4bc9-aff1-a8aaee683648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:49.332182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fda55-10ee-4bc9-aff1-a8aaee683648 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:49.332463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fda55-10ee-4bc9-aff1-a8aaee683648 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:49.332742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fda55-10ee-4bc9-aff1-a8aaee683648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:49.334429 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0f8fda55-10ee-4bc9-aff1-a8aaee683648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:49.453207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fda55-10ee-4bc9-aff1-a8aaee683648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:49.454592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fda55-10ee-4bc9-aff1-a8aaee683648 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:49.473081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 682/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:54.376728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd9e01-2309-40f9-a74b-0860a021a22b None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:54.377614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd9e01-2309-40f9-a74b-0860a021a22b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:54.377614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd9e01-2309-40f9-a74b-0860a021a22b None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:54.379023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 668/1350] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 05:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:50:54.386396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74548545-d6a8-487f-957b-3591fae810e0 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:54.386844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74548545-d6a8-487f-957b-3591fae810e0 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:54.386999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74548545-d6a8-487f-957b-3591fae810e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:54.521577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-74548545-d6a8-487f-957b-3591fae810e0 None None] MFA Rules not processed for user `dbfcf2e915c049f08a9e41c555401a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:54.522759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-74548545-d6a8-487f-957b-3591fae810e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:54.542720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:54.636981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 683/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 05:50:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:50:58.912379 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ece8f2-9ffd-4fe2-8b67-979d13ce6b77 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:58.912777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ece8f2-9ffd-4fe2-8b67-979d13ce6b77 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:58.913058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ece8f2-9ffd-4fe2-8b67-979d13ce6b77 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:58.914517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 669/1352] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 05:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:50:58.920082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf063d6-12e1-4fd4-9222-df90d421c633 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:58.920568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf063d6-12e1-4fd4-9222-df90d421c633 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:58.920648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf063d6-12e1-4fd4-9222-df90d421c633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:59.017796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-2cf063d6-12e1-4fd4-9222-df90d421c633 None None] MFA Rules not processed for user `dbfcf2e915c049f08a9e41c555401a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:59.019322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2cf063d6-12e1-4fd4-9222-df90d421c633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:59.118229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:59.160247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 684/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 05:50:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:00.222871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c770d-4aa2-48e7-8c4f-94fe099952b1 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:00.224833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c770d-4aa2-48e7-8c4f-94fe099952b1 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:00.224833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c770d-4aa2-48e7-8c4f-94fe099952b1 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:00.225201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 670/1354] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 05:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:00.230316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5031d-555d-425d-a3db-4662f6699ef0 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:00.230756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5031d-555d-425d-a3db-4662f6699ef0 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:00.230906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5031d-555d-425d-a3db-4662f6699ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:00.312862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-b6a5031d-555d-425d-a3db-4662f6699ef0 None None] MFA Rules not processed for user `dbfcf2e915c049f08a9e41c555401a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:00.314078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b6a5031d-555d-425d-a3db-4662f6699ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:00.419476 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 685/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 05:51:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:51:03.886590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee7133-bce4-41b6-a027-15ffa7093ab2 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:03.887523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee7133-bce4-41b6-a027-15ffa7093ab2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:03.888026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee7133-bce4-41b6-a027-15ffa7093ab2 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:03.890457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 671/1356] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:03.896282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8f0c-0acc-49f8-aab7-1793fbc2bc0d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:03.898450 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8f0c-0acc-49f8-aab7-1793fbc2bc0d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:03.898763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8f0c-0acc-49f8-aab7-1793fbc2bc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:03.937466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:03.970531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-c80f8f0c-0acc-49f8-aab7-1793fbc2bc0d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:03.971834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c80f8f0c-0acc-49f8-aab7-1793fbc2bc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:04.072767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 686/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:04.131161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e8362-56f7-41cf-b739-625c483ff8a2 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:04.131645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e8362-56f7-41cf-b739-625c483ff8a2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:04.132085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e8362-56f7-41cf-b739-625c483ff8a2 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:04.134016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 672/1358] 10.222.0.17 () {54 vars in 962 bytes} [Mon Jul 25 05:51:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:04.143734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22f89c-411a-45ec-a7d3-88807bba4e3d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:04.144234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22f89c-411a-45ec-a7d3-88807bba4e3d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:04.144671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22f89c-411a-45ec-a7d3-88807bba4e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:04.226366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-ff22f89c-411a-45ec-a7d3-88807bba4e3d None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:04.229174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ff22f89c-411a-45ec-a7d3-88807bba4e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:04.282128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:04.351275 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 687/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Jul 25 05:51:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:04.359359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e8362-56f7-41cf-b739-625c483ff8a2 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:04.361006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-440e8362-56f7-41cf-b739-625c483ff8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:04.516967 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e8362-56f7-41cf-b739-625c483ff8a2 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:04.597857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:04.639252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3cdc9-5847-4520-a469-399815402b8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:04.639979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3cdc9-5847-4520-a469-399815402b8c service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:04.640365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3cdc9-5847-4520-a469-399815402b8c service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:04.640743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3cdc9-5847-4520-a469-399815402b8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:04.642530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f7b3cdc9-5847-4520-a469-399815402b8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:04.753112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3cdc9-5847-4520-a469-399815402b8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:04.754776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3cdc9-5847-4520-a469-399815402b8c service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:04.774954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 673/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:51:07.055228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12a62e-92c7-4311-aced-fc0bbd1596ab None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:07.055849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12a62e-92c7-4311-aced-fc0bbd1596ab None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:07.055849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12a62e-92c7-4311-aced-fc0bbd1596ab None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:07.057681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 688/1361] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:07.063473 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cae5b-f984-4583-9474-cc205bff3197 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:07.064179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cae5b-f984-4583-9474-cc205bff3197 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:07.064179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cae5b-f984-4583-9474-cc205bff3197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:07.139258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-f07cae5b-f984-4583-9474-cc205bff3197 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:07.141173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f07cae5b-f984-4583-9474-cc205bff3197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:07.191259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:07.265812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 674/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:07.311072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12a62e-92c7-4311-aced-fc0bbd1596ab None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:07.312521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2b12a62e-92c7-4311-aced-fc0bbd1596ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:07.495826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12a62e-92c7-4311-aced-fc0bbd1596ab None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:07.521073 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:07.641116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89d772-45e3-4820-836d-fbba4522bf09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:07.641775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89d772-45e3-4820-836d-fbba4522bf09 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:07.642054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89d772-45e3-4820-836d-fbba4522bf09 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:07.642328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89d772-45e3-4820-836d-fbba4522bf09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:07.644043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4f89d772-45e3-4820-836d-fbba4522bf09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:07.736091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89d772-45e3-4820-836d-fbba4522bf09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:07.737718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89d772-45e3-4820-836d-fbba4522bf09 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:07.756864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 689/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:10.223213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edf128-34cd-439c-a701-e10b24146940 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:10.229391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edf128-34cd-439c-a701-e10b24146940 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:10.229391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edf128-34cd-439c-a701-e10b24146940 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:10.229856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 675/1364] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:10.231291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaba8db-c2f3-46a2-a1b3-0ee2a5144839 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:10.231714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaba8db-c2f3-46a2-a1b3-0ee2a5144839 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:10.232020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaba8db-c2f3-46a2-a1b3-0ee2a5144839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:10.252026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:10.323087 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1aaba8db-c2f3-46a2-a1b3-0ee2a5144839 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:10.324768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1aaba8db-c2f3-46a2-a1b3-0ee2a5144839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:10.435065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 690/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:10.482817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7d2f-2aa2-4564-8cb7-f765a262746b None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:10.483337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7d2f-2aa2-4564-8cb7-f765a262746b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:10.483775 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7d2f-2aa2-4564-8cb7-f765a262746b None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:10.485566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 676/1366] 10.222.0.17 () {54 vars in 962 bytes} [Mon Jul 25 05:51:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:10.491801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d312073-670d-4726-9882-e2a850f97c11 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:10.492115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d312073-670d-4726-9882-e2a850f97c11 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:10.492406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d312073-670d-4726-9882-e2a850f97c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:10.577002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9d312073-670d-4726-9882-e2a850f97c11 None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:10.578202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9d312073-670d-4726-9882-e2a850f97c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:10.618415 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:10.710094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 691/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Jul 25 05:51:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:10.719194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b7d2f-2aa2-4564-8cb7-f765a262746b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:10.720698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-012b7d2f-2aa2-4564-8cb7-f765a262746b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:10.874448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b7d2f-2aa2-4564-8cb7-f765a262746b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:10.897905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:10.999051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32f1af-e06c-4752-91f5-f737fcb258ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:10.999708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32f1af-e06c-4752-91f5-f737fcb258ef service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:10.999988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32f1af-e06c-4752-91f5-f737fcb258ef service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:11.000259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32f1af-e06c-4752-91f5-f737fcb258ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:11.001889 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7f32f1af-e06c-4752-91f5-f737fcb258ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:11.099841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32f1af-e06c-4752-91f5-f737fcb258ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:11.101472 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32f1af-e06c-4752-91f5-f737fcb258ef service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:11.121538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 677/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:12.044182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512e402-a7ee-4f18-b32c-b004355ec4be None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:12.044756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512e402-a7ee-4f18-b32c-b004355ec4be None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:12.044848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512e402-a7ee-4f18-b32c-b004355ec4be None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:12.046390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 692/1369] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:12.051405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f600c81-080a-4e25-8aa2-1c032b0d4ad3 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:12.051895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f600c81-080a-4e25-8aa2-1c032b0d4ad3 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:12.051979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f600c81-080a-4e25-8aa2-1c032b0d4ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:12.084023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:12.117715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2f600c81-080a-4e25-8aa2-1c032b0d4ad3 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:12.119056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2f600c81-080a-4e25-8aa2-1c032b0d4ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:12.220034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 678/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:12.562953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7d8c-9e7c-4685-b6ba-b456d7a6c420 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:12.562953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7d8c-9e7c-4685-b6ba-b456d7a6c420 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:12.562953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7d8c-9e7c-4685-b6ba-b456d7a6c420 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:12.565448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 693/1371] 10.222.0.17 () {54 vars in 964 bytes} [Mon Jul 25 05:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:12.568732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d40c9a-dd4b-4325-b05e-1e0d0ee6e97a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:12.569136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d40c9a-dd4b-4325-b05e-1e0d0ee6e97a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:12.569307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d40c9a-dd4b-4325-b05e-1e0d0ee6e97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:12.647212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c9d40c9a-dd4b-4325-b05e-1e0d0ee6e97a None None] MFA Rules not processed for user `6640e4c799514429a2fe7850369af4df`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:12.648427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c9d40c9a-dd4b-4325-b05e-1e0d0ee6e97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:12.698514 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:12.729843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 679/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Mon Jul 25 05:51:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:12.737317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c7d8c-9e7c-4685-b6ba-b456d7a6c420 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:12.738828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f74c7d8c-9e7c-4685-b6ba-b456d7a6c420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:12.854977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c7d8c-9e7c-4685-b6ba-b456d7a6c420 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:12.902130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:12.942837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4f3e3-a29b-4d50-adc0-e269eb11235f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:12.943501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4f3e3-a29b-4d50-adc0-e269eb11235f service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:12.943785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4f3e3-a29b-4d50-adc0-e269eb11235f service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:12.944064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4f3e3-a29b-4d50-adc0-e269eb11235f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:12.945709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c6a4f3e3-a29b-4d50-adc0-e269eb11235f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:13.063453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4f3e3-a29b-4d50-adc0-e269eb11235f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:13.065610 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4f3e3-a29b-4d50-adc0-e269eb11235f service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:13.087213 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 694/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:13.358177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1923e-1748-41a7-ab11-a59ca64aaa0c None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:13.358646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1923e-1748-41a7-ab11-a59ca64aaa0c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:13.358975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1923e-1748-41a7-ab11-a59ca64aaa0c None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:13.360412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 680/1374] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:13.365549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243be31-ec2c-48d2-860f-b47715d5a32f None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:13.365839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243be31-ec2c-48d2-860f-b47715d5a32f None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:13.366510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243be31-ec2c-48d2-860f-b47715d5a32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:13.431563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1243be31-ec2c-48d2-860f-b47715d5a32f None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:13.432837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1243be31-ec2c-48d2-860f-b47715d5a32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:13.462102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:13.550884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 695/1375] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:13.595390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1923e-1748-41a7-ab11-a59ca64aaa0c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:13.597039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6eb1923e-1748-41a7-ab11-a59ca64aaa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:13.742636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1923e-1748-41a7-ab11-a59ca64aaa0c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:13.795540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:13.856704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa2983-ee21-4de8-a87b-92a2e3c5065b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:13.857356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa2983-ee21-4de8-a87b-92a2e3c5065b service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:13.857642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa2983-ee21-4de8-a87b-92a2e3c5065b service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:13.857921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa2983-ee21-4de8-a87b-92a2e3c5065b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:13.859600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4afa2983-ee21-4de8-a87b-92a2e3c5065b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:13.951607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa2983-ee21-4de8-a87b-92a2e3c5065b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:13.953288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa2983-ee21-4de8-a87b-92a2e3c5065b service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:13.971896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 681/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:51:16.138256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217221f-5989-4bd6-8445-a17ffe324e7e None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:16.138785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217221f-5989-4bd6-8445-a17ffe324e7e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:16.138785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217221f-5989-4bd6-8445-a17ffe324e7e None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:16.140200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 696/1377] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:16.145746 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc849e-578a-4899-9b1d-39610928f320 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:16.146183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc849e-578a-4899-9b1d-39610928f320 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:16.146289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc849e-578a-4899-9b1d-39610928f320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:16.214623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8ecc849e-578a-4899-9b1d-39610928f320 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:16.216224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8ecc849e-578a-4899-9b1d-39610928f320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:16.223920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:16.329234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 682/1378] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:16.373761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217221f-5989-4bd6-8445-a17ffe324e7e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:16.375292 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0217221f-5989-4bd6-8445-a17ffe324e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:16.527112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217221f-5989-4bd6-8445-a17ffe324e7e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:16.530730 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:16.668979 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c08b86-b1dc-44b9-b0d6-52a3af5d8df8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:16.669674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c08b86-b1dc-44b9-b0d6-52a3af5d8df8 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:16.669955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c08b86-b1dc-44b9-b0d6-52a3af5d8df8 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:16.670228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c08b86-b1dc-44b9-b0d6-52a3af5d8df8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:16.672833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-80c08b86-b1dc-44b9-b0d6-52a3af5d8df8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:16.792698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c08b86-b1dc-44b9-b0d6-52a3af5d8df8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:16.794445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c08b86-b1dc-44b9-b0d6-52a3af5d8df8 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:16.816464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 697/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:18.999775 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fff46d-a503-48a8-81c5-7cbd28bec2fc None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:19.000264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fff46d-a503-48a8-81c5-7cbd28bec2fc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:19.000368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fff46d-a503-48a8-81c5-7cbd28bec2fc None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:19.001832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 683/1380] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:19.006829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cd679-c40b-4af6-93d2-01c6d76a1926 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:19.007241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cd679-c40b-4af6-93d2-01c6d76a1926 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:19.007405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cd679-c40b-4af6-93d2-01c6d76a1926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:19.017998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:19.077306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-cb1cd679-c40b-4af6-93d2-01c6d76a1926 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:19.078517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cb1cd679-c40b-4af6-93d2-01c6d76a1926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:19.185880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 698/1381] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:19.234017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fff46d-a503-48a8-81c5-7cbd28bec2fc None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:19.237547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a0fff46d-a503-48a8-81c5-7cbd28bec2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:19.386609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:19.397536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fff46d-a503-48a8-81c5-7cbd28bec2fc None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:19.514155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b1fb0-f03f-406c-8ca1-06390b76d80a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:19.514809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b1fb0-f03f-406c-8ca1-06390b76d80a service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:19.515102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b1fb0-f03f-406c-8ca1-06390b76d80a service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:19.515380 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b1fb0-f03f-406c-8ca1-06390b76d80a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:19.521295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-701b1fb0-f03f-406c-8ca1-06390b76d80a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:19.639810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b1fb0-f03f-406c-8ca1-06390b76d80a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:19.641573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b1fb0-f03f-406c-8ca1-06390b76d80a service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:19.652512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:19.661875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 684/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:21.923059 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e088906-4abd-4022-b5dc-2913732a73d8 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:21.923632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e088906-4abd-4022-b5dc-2913732a73d8 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:21.926374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e088906-4abd-4022-b5dc-2913732a73d8 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:21.927908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 699/1383] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:21.934289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72263ab-6b47-445f-b05e-4bba2910306c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:21.934923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72263ab-6b47-445f-b05e-4bba2910306c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:21.934923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72263ab-6b47-445f-b05e-4bba2910306c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:22.005005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-b72263ab-6b47-445f-b05e-4bba2910306c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:22.006440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b72263ab-6b47-445f-b05e-4bba2910306c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:22.120322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 685/1384] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:22.163770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e088906-4abd-4022-b5dc-2913732a73d8 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:22.165557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5e088906-4abd-4022-b5dc-2913732a73d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:22.258240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:22.340081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e088906-4abd-4022-b5dc-2913732a73d8 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:22.467972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726850e6-aef0-470e-83c7-8a99988104dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:22.468573 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726850e6-aef0-470e-83c7-8a99988104dd service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:22.468870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726850e6-aef0-470e-83c7-8a99988104dd service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:22.469150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726850e6-aef0-470e-83c7-8a99988104dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:22.470919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-726850e6-aef0-470e-83c7-8a99988104dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:22.526696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:22.569112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726850e6-aef0-470e-83c7-8a99988104dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:22.570886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726850e6-aef0-470e-83c7-8a99988104dd service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:22.590656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 700/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:25.378483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99394b-f4ae-493c-8826-cf0700481de8 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:25.379039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99394b-f4ae-493c-8826-cf0700481de8 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:25.379138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99394b-f4ae-493c-8826-cf0700481de8 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:25.380487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 686/1386] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:25.387360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892105a-ae33-414d-989f-72dbdf0e4809 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:25.387795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892105a-ae33-414d-989f-72dbdf0e4809 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:25.387959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892105a-ae33-414d-989f-72dbdf0e4809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:25.458135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d892105a-ae33-414d-989f-72dbdf0e4809 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:25.459437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d892105a-ae33-414d-989f-72dbdf0e4809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:25.559667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 701/1387] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:25.621608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99394b-f4ae-493c-8826-cf0700481de8 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:25.623649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bc99394b-f4ae-493c-8826-cf0700481de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:25.703107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:25.775624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99394b-f4ae-493c-8826-cf0700481de8 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:25.916894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba161b6-5910-458d-9f00-9ab52b5c6b0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:25.917692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba161b6-5910-458d-9f00-9ab52b5c6b0d service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:25.918109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba161b6-5910-458d-9f00-9ab52b5c6b0d service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:25.918562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba161b6-5910-458d-9f00-9ab52b5c6b0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:25.920624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-eba161b6-5910-458d-9f00-9ab52b5c6b0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:25.967108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:26.033269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba161b6-5910-458d-9f00-9ab52b5c6b0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:26.035405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba161b6-5910-458d-9f00-9ab52b5c6b0d service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:26.057849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 687/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:51:28.236749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e9a03-a71f-4415-842a-bb1ea59a485c None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.237294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e9a03-a71f-4415-842a-bb1ea59a485c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.237294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e9a03-a71f-4415-842a-bb1ea59a485c None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.238814 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 702/1389] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:28.243774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c95f6-82e5-4ff0-ae20-29836a476a8d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.244160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c95f6-82e5-4ff0-ae20-29836a476a8d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.244329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c95f6-82e5-4ff0-ae20-29836a476a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.310049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e13c95f6-82e5-4ff0-ae20-29836a476a8d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:28.311342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e13c95f6-82e5-4ff0-ae20-29836a476a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:28.407890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 688/1390] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:28.430970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c557201-3d7f-4f47-909c-6bdc8e6798e9 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.431333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c557201-3d7f-4f47-909c-6bdc8e6798e9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.431707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c557201-3d7f-4f47-909c-6bdc8e6798e9 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.433227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 703/1391] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:28.438926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e62511-83e5-469e-b7fc-0a32d18acdef None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.439489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e62511-83e5-469e-b7fc-0a32d18acdef None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.439489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e62511-83e5-469e-b7fc-0a32d18acdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.452009 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c557201-3d7f-4f47-909c-6bdc8e6798e9 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:28.453614 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3c557201-3d7f-4f47-909c-6bdc8e6798e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:28.454238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:28.484760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:28.506214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-22e62511-83e5-469e-b7fc-0a32d18acdef None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:28.507399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-22e62511-83e5-469e-b7fc-0a32d18acdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:28.600456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c557201-3d7f-4f47-909c-6bdc8e6798e9 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:28.603292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 689/1392] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:28.659385 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376bc0a-6d76-4ed9-b171-1d5ffc60a4d1 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.659696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376bc0a-6d76-4ed9-b171-1d5ffc60a4d1 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.659980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376bc0a-6d76-4ed9-b171-1d5ffc60a4d1 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.661359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 690/1393] 10.222.0.17 () {54 vars in 966 bytes} [Mon Jul 25 05:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:51:28.666016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd8551-db29-40ea-8749-3bb19aec88e2 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.666297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd8551-db29-40ea-8749-3bb19aec88e2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.666606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd8551-db29-40ea-8749-3bb19aec88e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.717339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:28.728964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08d932-e5c7-4e3a-9885-03adb0337402 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:28.731935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08d932-e5c7-4e3a-9885-03adb0337402 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.731935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08d932-e5c7-4e3a-9885-03adb0337402 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.731935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08d932-e5c7-4e3a-9885-03adb0337402 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.731935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-db08d932-e5c7-4e3a-9885-03adb0337402 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:28.746439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-48bd8551-db29-40ea-8749-3bb19aec88e2 None None] MFA Rules not processed for user `6e12361da5f849ee97607ca68ee6ef10`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:28.748279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-48bd8551-db29-40ea-8749-3bb19aec88e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:28.784169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:28.823516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08d932-e5c7-4e3a-9885-03adb0337402 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:28.825148 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08d932-e5c7-4e3a-9885-03adb0337402 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:28.828117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 691/1394] 10.222.0.17 () {58 vars in 1063 bytes} [Mon Jul 25 05:51:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:51:28.835575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd8551-db29-40ea-8749-3bb19aec88e2 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:28.842989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd8551-db29-40ea-8749-3bb19aec88e2 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:28.843611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 704/1395] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:51:28.933674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39d878-c5a7-487a-9919-029f02d7fcb9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e12361da5f849ee97607ca68ee6ef10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:28.934327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39d878-c5a7-487a-9919-029f02d7fcb9 service glance] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.934661 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39d878-c5a7-487a-9919-029f02d7fcb9 service glance] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.934933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39d878-c5a7-487a-9919-029f02d7fcb9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.937255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bf39d878-c5a7-487a-9919-029f02d7fcb9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:29.053158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:29.056911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39d878-c5a7-487a-9919-029f02d7fcb9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:29.058435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39d878-c5a7-487a-9919-029f02d7fcb9 service glance] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:29.078812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 692/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:51:29.983175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e879ec-2d37-4e06-a96a-2028b4ef36cc None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:29.983651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e879ec-2d37-4e06-a96a-2028b4ef36cc None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:29.983862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e879ec-2d37-4e06-a96a-2028b4ef36cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:30.069503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a5e879ec-2d37-4e06-a96a-2028b4ef36cc None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:30.070800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a5e879ec-2d37-4e06-a96a-2028b4ef36cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:30.130716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:30.158086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 705/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 05:51:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:30.205356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a637c-fd05-464e-9470-86d6f4bea098 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:30.205875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a637c-fd05-464e-9470-86d6f4bea098 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:30.205976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a637c-fd05-464e-9470-86d6f4bea098 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:30.207354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 693/1398] 10.222.0.17 () {54 vars in 943 bytes} [Mon Jul 25 05:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:30.217066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa12231-cac9-4547-a927-26d1e0f36101 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:30.217483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa12231-cac9-4547-a927-26d1e0f36101 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:30.217770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa12231-cac9-4547-a927-26d1e0f36101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:30.292770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-afa12231-cac9-4547-a927-26d1e0f36101 None None] MFA Rules not processed for user `5c359fc0168141fb8da3f953884fd2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:30.293977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-afa12231-cac9-4547-a927-26d1e0f36101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:30.400777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 706/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Mon Jul 25 05:51:30 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:30.410326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a637c-fd05-464e-9470-86d6f4bea098 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:30.411864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2f2a637c-fd05-464e-9470-86d6f4bea098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:30.552657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a637c-fd05-464e-9470-86d6f4bea098 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:30.628813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:30.663089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf95ff-4429-4740-8f62-1e1a92c6c7d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:30.663750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf95ff-4429-4740-8f62-1e1a92c6c7d1 service swift] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:30.664034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf95ff-4429-4740-8f62-1e1a92c6c7d1 service swift] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:30.664805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf95ff-4429-4740-8f62-1e1a92c6c7d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:30.666578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0ecf95ff-4429-4740-8f62-1e1a92c6c7d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:30.752749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf95ff-4429-4740-8f62-1e1a92c6c7d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:30.754273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf95ff-4429-4740-8f62-1e1a92c6c7d1 service swift] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:30.762461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 694/1400] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 05:51:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:51:30.989632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9502256-cd7b-49d8-969a-7b319fee902c None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:30.990169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9502256-cd7b-49d8-969a-7b319fee902c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:30.990169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9502256-cd7b-49d8-969a-7b319fee902c None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:30.991635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 707/1401] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:30.996927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268e992-fd19-472d-8064-76fceca98e5f None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:30.997237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268e992-fd19-472d-8064-76fceca98e5f None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:30.997555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268e992-fd19-472d-8064-76fceca98e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:31.069141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-4268e992-fd19-472d-8064-76fceca98e5f None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:31.071830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4268e992-fd19-472d-8064-76fceca98e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:31.124813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:31.165626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 695/1402] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:31.209683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9502256-cd7b-49d8-969a-7b319fee902c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:31.211151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c9502256-cd7b-49d8-969a-7b319fee902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:31.257086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:31.381664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9502256-cd7b-49d8-969a-7b319fee902c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:31.507741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:31.510617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ada6c-c5ab-4cd1-bb12-d49c8c6cb1a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:31.511277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ada6c-c5ab-4cd1-bb12-d49c8c6cb1a3 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:31.511563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ada6c-c5ab-4cd1-bb12-d49c8c6cb1a3 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:31.514237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ada6c-c5ab-4cd1-bb12-d49c8c6cb1a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:31.516129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-083ada6c-c5ab-4cd1-bb12-d49c8c6cb1a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:31.616258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ada6c-c5ab-4cd1-bb12-d49c8c6cb1a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:31.617885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ada6c-c5ab-4cd1-bb12-d49c8c6cb1a3 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:31.636963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 708/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:33.874679 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718edbb7-33ab-4e4e-8ad1-c163c0f60bb9 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:33.875286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718edbb7-33ab-4e4e-8ad1-c163c0f60bb9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:33.875637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718edbb7-33ab-4e4e-8ad1-c163c0f60bb9 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:33.877327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 696/1404] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:33.882579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32af44-5e11-44ca-ac3b-c71e368233e5 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:33.883023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32af44-5e11-44ca-ac3b-c71e368233e5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:33.883176 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32af44-5e11-44ca-ac3b-c71e368233e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:33.954917 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-8a32af44-5e11-44ca-ac3b-c71e368233e5 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:33.956552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8a32af44-5e11-44ca-ac3b-c71e368233e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:34.031500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:34.064565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 709/1405] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:34.108901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718edbb7-33ab-4e4e-8ad1-c163c0f60bb9 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:34.110555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-718edbb7-33ab-4e4e-8ad1-c163c0f60bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:34.276574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718edbb7-33ab-4e4e-8ad1-c163c0f60bb9 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:34.322324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:34.402933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2677be-4896-4908-82be-feee82c35ac1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:34.403699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2677be-4896-4908-82be-feee82c35ac1 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:34.404568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2677be-4896-4908-82be-feee82c35ac1 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:34.404849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2677be-4896-4908-82be-feee82c35ac1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:34.406801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-df2677be-4896-4908-82be-feee82c35ac1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:34.514819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2677be-4896-4908-82be-feee82c35ac1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:34.516718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2677be-4896-4908-82be-feee82c35ac1 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:34.535655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 697/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:36.838363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994c168-6a7c-49a3-86c9-ac7b7f5b18b7 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:36.838961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994c168-6a7c-49a3-86c9-ac7b7f5b18b7 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:36.838961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994c168-6a7c-49a3-86c9-ac7b7f5b18b7 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:36.840357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 710/1407] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:36.846120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a88de-7140-4fc3-8806-73734439505f None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:36.846566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a88de-7140-4fc3-8806-73734439505f None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:36.846780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a88de-7140-4fc3-8806-73734439505f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:36.922309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:36.930627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-035a88de-7140-4fc3-8806-73734439505f None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:36.931893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-035a88de-7140-4fc3-8806-73734439505f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:37.044158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 698/1408] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:37.136140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994c168-6a7c-49a3-86c9-ac7b7f5b18b7 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:37.136140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0994c168-6a7c-49a3-86c9-ac7b7f5b18b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:37.285453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994c168-6a7c-49a3-86c9-ac7b7f5b18b7 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:37.358108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:37.444437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b47308-012d-451d-b899-32a4cf8ffd71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:37.445109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47308-012d-451d-b899-32a4cf8ffd71 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:37.445451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47308-012d-451d-b899-32a4cf8ffd71 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:37.445802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47308-012d-451d-b899-32a4cf8ffd71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:37.447479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-32b47308-012d-451d-b899-32a4cf8ffd71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:37.548524 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b47308-012d-451d-b899-32a4cf8ffd71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:37.550243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b47308-012d-451d-b899-32a4cf8ffd71 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:37.572796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 711/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:47.042106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=44 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:47.570224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=73 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:54.768109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883ddd2-f855-4f10-b507-a7442a3dd5d8 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:54.768704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883ddd2-f855-4f10-b507-a7442a3dd5d8 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:54.768704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883ddd2-f855-4f10-b507-a7442a3dd5d8 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:54.770033 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 699/1410] 10.222.0.17 () {54 vars in 918 bytes} [Mon Jul 25 05:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:54.775129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620dcf4-b4c9-4596-b3f7-8f8aa29733a9 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:54.775579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620dcf4-b4c9-4596-b3f7-8f8aa29733a9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:54.775718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620dcf4-b4c9-4596-b3f7-8f8aa29733a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:54.850388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a620dcf4-b4c9-4596-b3f7-8f8aa29733a9 None None] MFA Rules not processed for user `dbfcf2e915c049f08a9e41c555401a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:54.851638 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a620dcf4-b4c9-4596-b3f7-8f8aa29733a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:54.960941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 712/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Jul 25 05:51:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:54.975197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed085e-61b9-4a3e-9366-6de2655fca26 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:54.975490 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed085e-61b9-4a3e-9366-6de2655fca26 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:54.975772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed085e-61b9-4a3e-9366-6de2655fca26 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:54.977153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 700/1412] 10.222.0.17 () {54 vars in 918 bytes} [Mon Jul 25 05:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:54.981920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ec53-88cb-4153-a539-72ee70814d2c None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:54.982214 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ec53-88cb-4153-a539-72ee70814d2c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:54.982516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ec53-88cb-4153-a539-72ee70814d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:55.051966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:55.054371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-10f4ec53-88cb-4153-a539-72ee70814d2c None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:55.055599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-10f4ec53-88cb-4153-a539-72ee70814d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:55.169877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 713/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Jul 25 05:51:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:57.659500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960f84-1b1b-4274-91cc-e54c2e992fd8 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:57.660040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960f84-1b1b-4274-91cc-e54c2e992fd8 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:57.660040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960f84-1b1b-4274-91cc-e54c2e992fd8 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:57.661472 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:57.666712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07091d-7b63-496c-b27f-759a57deb53d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:57.667136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07091d-7b63-496c-b27f-759a57deb53d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:57.667269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07091d-7b63-496c-b27f-759a57deb53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:57.730771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-6d07091d-7b63-496c-b27f-759a57deb53d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:57.732034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6d07091d-7b63-496c-b27f-759a57deb53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:57.786022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:57.827700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:57.868935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb960f84-1b1b-4274-91cc-e54c2e992fd8 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:57.870447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-fb960f84-1b1b-4274-91cc-e54c2e992fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:58.013085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb960f84-1b1b-4274-91cc-e54c2e992fd8 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:58.085052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:58.125963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5dfc48-c561-4fdb-b58b-d5031c5c2826 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:58.126643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dfc48-c561-4fdb-b58b-d5031c5c2826 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:58.126924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dfc48-c561-4fdb-b58b-d5031c5c2826 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:58.127197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5dfc48-c561-4fdb-b58b-d5031c5c2826 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:58.128807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bc5dfc48-c561-4fdb-b58b-d5031c5c2826 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:58.215585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dfc48-c561-4fdb-b58b-d5031c5c2826 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:58.217179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5dfc48-c561-4fdb-b58b-d5031c5c2826 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:58.235218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 702/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:59.133516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5dbc5-b3ab-4ad7-ae55-0bbf7525088e None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:59.134083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5dbc5-b3ab-4ad7-ae55-0bbf7525088e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:59.134083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5dbc5-b3ab-4ad7-ae55-0bbf7525088e None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:59.135516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:59.140514 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b47ff1-3e25-4e05-9d62-567bc456cbbd None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:59.140912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b47ff1-3e25-4e05-9d62-567bc456cbbd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:59.141070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b47ff1-3e25-4e05-9d62-567bc456cbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:59.215433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-57b47ff1-3e25-4e05-9d62-567bc456cbbd None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:59.217062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-57b47ff1-3e25-4e05-9d62-567bc456cbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:59.243388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:59.311640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:59.460724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5dbc5-b3ab-4ad7-ae55-0bbf7525088e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:59.462228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f4b5dbc5-b3ab-4ad7-ae55-0bbf7525088e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:59.611710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5dbc5-b3ab-4ad7-ae55-0bbf7525088e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:59.650583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:59.722957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa42ac-1af4-42b4-97a2-e9212ca655dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:59.723608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa42ac-1af4-42b4-97a2-e9212ca655dc service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:59.723891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa42ac-1af4-42b4-97a2-e9212ca655dc service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:59.724167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa42ac-1af4-42b4-97a2-e9212ca655dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:59.725817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-11aa42ac-1af4-42b4-97a2-e9212ca655dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:59.811685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa42ac-1af4-42b4-97a2-e9212ca655dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:59.813304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa42ac-1af4-42b4-97a2-e9212ca655dc service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:59.831238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 716/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:00.712835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89c4f0-b01e-4646-94c1-b8aea1041aec None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:00.713383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89c4f0-b01e-4646-94c1-b8aea1041aec None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:00.713383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89c4f0-b01e-4646-94c1-b8aea1041aec None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:00.714814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:00.720194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dc4e1-68aa-4d61-bb17-c1ece1b25459 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:00.720622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dc4e1-68aa-4d61-bb17-c1ece1b25459 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:00.720776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dc4e1-68aa-4d61-bb17-c1ece1b25459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:00.777902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:00.802063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-214dc4e1-68aa-4d61-bb17-c1ece1b25459 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:00.803313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-214dc4e1-68aa-4d61-bb17-c1ece1b25459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:00.898568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:01.051782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89c4f0-b01e-4646-94c1-b8aea1041aec None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:01.053202 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4c89c4f0-b01e-4646-94c1-b8aea1041aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:01.195270 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89c4f0-b01e-4646-94c1-b8aea1041aec None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:01.207644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:01.306658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b2287-17b0-4d23-9f94-068296d06e3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:01.307217 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b2287-17b0-4d23-9f94-068296d06e3b service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:01.307497 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b2287-17b0-4d23-9f94-068296d06e3b service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:01.307766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b2287-17b0-4d23-9f94-068296d06e3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:01.309376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f17b2287-17b0-4d23-9f94-068296d06e3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:01.396822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b2287-17b0-4d23-9f94-068296d06e3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:01.398407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b2287-17b0-4d23-9f94-068296d06e3b service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:01.416681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 705/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:02.281762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff340e12-5b01-47c7-bf20-6d5fd60dc6dd None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:02.282583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff340e12-5b01-47c7-bf20-6d5fd60dc6dd None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:02.282583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff340e12-5b01-47c7-bf20-6d5fd60dc6dd None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:02.283990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:02.288931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec60e6-b7a4-4736-bff9-95e05301aeb6 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:02.289349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec60e6-b7a4-4736-bff9-95e05301aeb6 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:02.289461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec60e6-b7a4-4736-bff9-95e05301aeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:02.308430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:02.364981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-89ec60e6-b7a4-4736-bff9-95e05301aeb6 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:02.366188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-89ec60e6-b7a4-4736-bff9-95e05301aeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:02.459407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:02.608516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff340e12-5b01-47c7-bf20-6d5fd60dc6dd None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:02.610037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ff340e12-5b01-47c7-bf20-6d5fd60dc6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:02.717128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:02.869410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff340e12-5b01-47c7-bf20-6d5fd60dc6dd None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:02.980192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb8824-3ee5-4076-a44c-871c511d7819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:02.980844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb8824-3ee5-4076-a44c-871c511d7819 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:02.981130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb8824-3ee5-4076-a44c-871c511d7819 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:02.981406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb8824-3ee5-4076-a44c-871c511d7819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:02.983086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c4cb8824-3ee5-4076-a44c-871c511d7819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:03.056470 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:03.069843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb8824-3ee5-4076-a44c-871c511d7819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:03.071482 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb8824-3ee5-4076-a44c-871c511d7819 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:03.090244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 719/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:03.956577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540858-5e4f-43f6-bacc-482476bfb49c None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:03.957128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540858-5e4f-43f6-bacc-482476bfb49c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:03.957128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540858-5e4f-43f6-bacc-482476bfb49c None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:03.958545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:03.963678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17f52a-380b-46b8-8097-ba96f1d2883d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:03.964115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17f52a-380b-46b8-8097-ba96f1d2883d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:03.964246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17f52a-380b-46b8-8097-ba96f1d2883d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:04.037093 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-2e17f52a-380b-46b8-8097-ba96f1d2883d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:04.038302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2e17f52a-380b-46b8-8097-ba96f1d2883d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:04.133523 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:04.285545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a540858-5e4f-43f6-bacc-482476bfb49c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:04.287005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2a540858-5e4f-43f6-bacc-482476bfb49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:04.369411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:04.430708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a540858-5e4f-43f6-bacc-482476bfb49c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:04.541614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b72535-bc8c-4966-85f4-76f9c52650d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:04.542182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b72535-bc8c-4966-85f4-76f9c52650d5 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:04.542463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b72535-bc8c-4966-85f4-76f9c52650d5 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:04.542768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b72535-bc8c-4966-85f4-76f9c52650d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:04.544419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-22b72535-bc8c-4966-85f4-76f9c52650d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:04.592055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:04.632025 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b72535-bc8c-4966-85f4-76f9c52650d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:04.633613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b72535-bc8c-4966-85f4-76f9c52650d5 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:04.652693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 708/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:05.505135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f90fa5-8dae-45a9-ad52-786cad3e667d None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:05.505699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f90fa5-8dae-45a9-ad52-786cad3e667d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:05.505699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f90fa5-8dae-45a9-ad52-786cad3e667d None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:05.507138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:05.511999 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800f24d-3e53-427b-97ee-7c18a68bff88 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:05.512430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800f24d-3e53-427b-97ee-7c18a68bff88 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:05.512536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800f24d-3e53-427b-97ee-7c18a68bff88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:05.588784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-b800f24d-3e53-427b-97ee-7c18a68bff88 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:05.589910 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b800f24d-3e53-427b-97ee-7c18a68bff88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:05.683798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:05.834181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f90fa5-8dae-45a9-ad52-786cad3e667d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:05.835764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-79f90fa5-8dae-45a9-ad52-786cad3e667d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:05.885066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:05.980284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f90fa5-8dae-45a9-ad52-786cad3e667d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:06.089616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d621d-b044-456f-9446-e282b3bae2d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:06.090252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d621d-b044-456f-9446-e282b3bae2d9 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:06.090552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d621d-b044-456f-9446-e282b3bae2d9 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:06.090836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d621d-b044-456f-9446-e282b3bae2d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:06.093437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ed0d621d-b044-456f-9446-e282b3bae2d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:06.100148 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:06.184425 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d621d-b044-456f-9446-e282b3bae2d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:06.186483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d621d-b044-456f-9446-e282b3bae2d9 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:06.204394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 722/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:07.069126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79b069-11f5-4e92-b008-506127a3544e None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:07.069715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79b069-11f5-4e92-b008-506127a3544e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:07.069715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79b069-11f5-4e92-b008-506127a3544e None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:07.071102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:07.076141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860aec5-d181-4c80-85e5-9ff1181de1a3 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:07.076569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860aec5-d181-4c80-85e5-9ff1181de1a3 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:07.076680 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860aec5-d181-4c80-85e5-9ff1181de1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:07.150059 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-3860aec5-d181-4c80-85e5-9ff1181de1a3 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:07.151343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3860aec5-d181-4c80-85e5-9ff1181de1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:07.218057 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:07.248458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:07.400125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79b069-11f5-4e92-b008-506127a3544e None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:07.401656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8a79b069-11f5-4e92-b008-506127a3544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:07.420168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:07.549443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79b069-11f5-4e92-b008-506127a3544e None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:07.642080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:07.667722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e301de-d66e-4f5f-8b11-8dae81cf4c27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:07.668382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e301de-d66e-4f5f-8b11-8dae81cf4c27 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:07.668837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e301de-d66e-4f5f-8b11-8dae81cf4c27 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:07.669392 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e301de-d66e-4f5f-8b11-8dae81cf4c27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:07.671078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e0e301de-d66e-4f5f-8b11-8dae81cf4c27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:07.758116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e301de-d66e-4f5f-8b11-8dae81cf4c27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:07.759741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e301de-d66e-4f5f-8b11-8dae81cf4c27 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:07.778183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 711/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:08.662194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36162e76-cf66-4f7a-9fbc-7168787abaed None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:08.662794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36162e76-cf66-4f7a-9fbc-7168787abaed None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:08.662794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36162e76-cf66-4f7a-9fbc-7168787abaed None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:08.664168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:52:08.669051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ec334-026b-47da-be87-d8e3f1ed15e4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:08.670024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ec334-026b-47da-be87-d8e3f1ed15e4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:08.670139 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ec334-026b-47da-be87-d8e3f1ed15e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:08.744233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2c3ec334-026b-47da-be87-d8e3f1ed15e4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:08.745453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2c3ec334-026b-47da-be87-d8e3f1ed15e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:08.786078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:08.843564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:08.997007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36162e76-cf66-4f7a-9fbc-7168787abaed None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:08.998456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-36162e76-cf66-4f7a-9fbc-7168787abaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:09.139927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36162e76-cf66-4f7a-9fbc-7168787abaed None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:09.194911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:09.252871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eab4a-49e5-4a34-9ebb-f388a245f55d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:09.253440 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eab4a-49e5-4a34-9ebb-f388a245f55d service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:09.253714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eab4a-49e5-4a34-9ebb-f388a245f55d service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:09.253982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eab4a-49e5-4a34-9ebb-f388a245f55d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:09.255644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cf1eab4a-49e5-4a34-9ebb-f388a245f55d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:09.343165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1eab4a-49e5-4a34-9ebb-f388a245f55d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:09.344765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1eab4a-49e5-4a34-9ebb-f388a245f55d service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:09.362162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 725/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:10.228158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1828b-7f24-477a-90b7-4c421aab6227 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:10.228702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1828b-7f24-477a-90b7-4c421aab6227 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:10.228702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1828b-7f24-477a-90b7-4c421aab6227 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:10.230019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:10.235534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed6b48-bf4e-4e4f-bed2-533a7500f1c6 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:10.235927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed6b48-bf4e-4e4f-bed2-533a7500f1c6 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:10.236096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed6b48-bf4e-4e4f-bed2-533a7500f1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:10.311432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-91ed6b48-bf4e-4e4f-bed2-533a7500f1c6 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:10.312628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-91ed6b48-bf4e-4e4f-bed2-533a7500f1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:10.314767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:10.408373 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:10.555560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1828b-7f24-477a-90b7-4c421aab6227 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:10.556984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-15c1828b-7f24-477a-90b7-4c421aab6227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:10.708075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1828b-7f24-477a-90b7-4c421aab6227 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:10.735837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:10.825694 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f0830-0f59-45de-a2f3-d4a6cecdf073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:10.826277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f0830-0f59-45de-a2f3-d4a6cecdf073 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:10.826588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f0830-0f59-45de-a2f3-d4a6cecdf073 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:10.827263 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f0830-0f59-45de-a2f3-d4a6cecdf073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:10.828890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-111f0830-0f59-45de-a2f3-d4a6cecdf073 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:10.917852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f0830-0f59-45de-a2f3-d4a6cecdf073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:10.920027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f0830-0f59-45de-a2f3-d4a6cecdf073 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:10.938360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 714/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:52:12.869170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf56f9-34c1-4fb6-8601-6fd876f78f82 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:12.869739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf56f9-34c1-4fb6-8601-6fd876f78f82 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:12.869739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf56f9-34c1-4fb6-8601-6fd876f78f82 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:12.871159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:12.876983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849854-fa66-404a-b9f7-fb51174e334b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:12.877410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849854-fa66-404a-b9f7-fb51174e334b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:12.877562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849854-fa66-404a-b9f7-fb51174e334b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:12.917413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:12.943464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e2849854-fa66-404a-b9f7-fb51174e334b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:12.944687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e2849854-fa66-404a-b9f7-fb51174e334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:13.039783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:13.079459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf56f9-34c1-4fb6-8601-6fd876f78f82 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:13.081090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5bdf56f9-34c1-4fb6-8601-6fd876f78f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:13.182532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf56f9-34c1-4fb6-8601-6fd876f78f82 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:13.214126 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:13.255111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55bf02-76f4-4b43-a9a0-99150c0ab2a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:13.255763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55bf02-76f4-4b43-a9a0-99150c0ab2a1 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:13.256094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55bf02-76f4-4b43-a9a0-99150c0ab2a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:13.256311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55bf02-76f4-4b43-a9a0-99150c0ab2a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:13.257933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cd55bf02-76f4-4b43-a9a0-99150c0ab2a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:13.363396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55bf02-76f4-4b43-a9a0-99150c0ab2a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:13.364832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55bf02-76f4-4b43-a9a0-99150c0ab2a1 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:13.383389 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 728/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:15.547435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2dfa5-0aef-4e74-9d4e-be95dcdc44cd None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:15.548036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2dfa5-0aef-4e74-9d4e-be95dcdc44cd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:15.548036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2dfa5-0aef-4e74-9d4e-be95dcdc44cd None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:15.549355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:15.554552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e16c93-4892-4bb1-99b9-ada48d379f0b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:15.554940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e16c93-4892-4bb1-99b9-ada48d379f0b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:15.555116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e16c93-4892-4bb1-99b9-ada48d379f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:15.620811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-36e16c93-4892-4bb1-99b9-ada48d379f0b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:15.622037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-36e16c93-4892-4bb1-99b9-ada48d379f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:15.634459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:15.717191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:15.755943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2dfa5-0aef-4e74-9d4e-be95dcdc44cd None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:15.758254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-aae2dfa5-0aef-4e74-9d4e-be95dcdc44cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:15.860819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2dfa5-0aef-4e74-9d4e-be95dcdc44cd None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:15.873604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:15.936684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944851a-bd09-4139-940e-d62b048ce21f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:15.937340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944851a-bd09-4139-940e-d62b048ce21f service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:15.937621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944851a-bd09-4139-940e-d62b048ce21f service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:15.937904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944851a-bd09-4139-940e-d62b048ce21f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:15.939598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6944851a-bd09-4139-940e-d62b048ce21f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:16.047351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944851a-bd09-4139-940e-d62b048ce21f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:16.048781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944851a-bd09-4139-940e-d62b048ce21f service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:16.066841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 717/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:18.469329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208f810-1a87-44f9-bfb5-7f38d751ceda None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:18.469930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208f810-1a87-44f9-bfb5-7f38d751ceda None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:18.469930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208f810-1a87-44f9-bfb5-7f38d751ceda None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:18.471321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:18.476556 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca516eb-d78b-4097-b10c-37d107026839 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:18.476994 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca516eb-d78b-4097-b10c-37d107026839 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:18.477110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca516eb-d78b-4097-b10c-37d107026839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:18.524083 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:18.541876 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-aca516eb-d78b-4097-b10c-37d107026839 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:18.543145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-aca516eb-d78b-4097-b10c-37d107026839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:18.639222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:18.678200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208f810-1a87-44f9-bfb5-7f38d751ceda None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:18.679788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f208f810-1a87-44f9-bfb5-7f38d751ceda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:18.780593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208f810-1a87-44f9-bfb5-7f38d751ceda None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:18.819850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:18.853243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1e545-201d-4530-b70f-dc4f40719228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:18.853879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1e545-201d-4530-b70f-dc4f40719228 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:18.854155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1e545-201d-4530-b70f-dc4f40719228 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:18.854425 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1e545-201d-4530-b70f-dc4f40719228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:18.856051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-eac1e545-201d-4530-b70f-dc4f40719228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:18.961752 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1e545-201d-4530-b70f-dc4f40719228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:18.963224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1e545-201d-4530-b70f-dc4f40719228 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:18.982295 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 731/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:21.130304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2632e-b724-44a6-8af5-df81852424f4 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:21.130888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2632e-b724-44a6-8af5-df81852424f4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:21.130888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2632e-b724-44a6-8af5-df81852424f4 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:21.132278 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:21.137700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b30bfa-7e81-49e6-9f58-9da81f778468 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:21.138131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b30bfa-7e81-49e6-9f58-9da81f778468 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:21.138373 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b30bfa-7e81-49e6-9f58-9da81f778468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:21.207402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-90b30bfa-7e81-49e6-9f58-9da81f778468 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:21.208541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-90b30bfa-7e81-49e6-9f58-9da81f778468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:21.229116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:21.304861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:21.344280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2632e-b724-44a6-8af5-df81852424f4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:21.345805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1fc2632e-b724-44a6-8af5-df81852424f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:21.445725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2632e-b724-44a6-8af5-df81852424f4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:21.465564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:21.529186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5b0d9-bc40-4066-b314-b9346f10d254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:21.529881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5b0d9-bc40-4066-b314-b9346f10d254 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:21.530160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5b0d9-bc40-4066-b314-b9346f10d254 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:21.530437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5b0d9-bc40-4066-b314-b9346f10d254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:21.532101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0fd5b0d9-bc40-4066-b314-b9346f10d254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:21.641008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5b0d9-bc40-4066-b314-b9346f10d254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:21.642449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5b0d9-bc40-4066-b314-b9346f10d254 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:21.663052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 720/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:23.729417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b494f5e-95ef-4827-bb39-fa0387c37169 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:23.729982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b494f5e-95ef-4827-bb39-fa0387c37169 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:23.729982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b494f5e-95ef-4827-bb39-fa0387c37169 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:23.731431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:52:23.736582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ecc2d-478c-4280-ac85-d3ca162ad730 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:23.737043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ecc2d-478c-4280-ac85-d3ca162ad730 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:23.737140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ecc2d-478c-4280-ac85-d3ca162ad730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:23.798365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:23.800944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-894ecc2d-478c-4280-ac85-d3ca162ad730 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:23.802142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-894ecc2d-478c-4280-ac85-d3ca162ad730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:23.896278 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:23.935632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b494f5e-95ef-4827-bb39-fa0387c37169 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:23.937266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4b494f5e-95ef-4827-bb39-fa0387c37169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:24.038283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b494f5e-95ef-4827-bb39-fa0387c37169 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:24.086934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:24.111330 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63580e9f-5cb6-412e-9590-37d6befe1c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:24.111982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63580e9f-5cb6-412e-9590-37d6befe1c93 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:24.112264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63580e9f-5cb6-412e-9590-37d6befe1c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:24.112540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63580e9f-5cb6-412e-9590-37d6befe1c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:24.114216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-63580e9f-5cb6-412e-9590-37d6befe1c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:24.232642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63580e9f-5cb6-412e-9590-37d6befe1c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:24.234087 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63580e9f-5cb6-412e-9590-37d6befe1c93 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:24.253743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 734/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:26.361354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519a87f-58a3-4da4-a258-022698c9302b None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:26.362073 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519a87f-58a3-4da4-a258-022698c9302b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:26.362073 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519a87f-58a3-4da4-a258-022698c9302b None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:26.363471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:26.368891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6523e73-c525-4a64-808e-8a0e946baee8 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:26.369332 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6523e73-c525-4a64-808e-8a0e946baee8 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:26.369427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6523e73-c525-4a64-808e-8a0e946baee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:26.432252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d6523e73-c525-4a64-808e-8a0e946baee8 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:26.433446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d6523e73-c525-4a64-808e-8a0e946baee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:26.461142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:26.527282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:26.565188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519a87f-58a3-4da4-a258-022698c9302b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:26.566756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6519a87f-58a3-4da4-a258-022698c9302b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:26.667881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519a87f-58a3-4da4-a258-022698c9302b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:26.694721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:26.740066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4585e8-0fd1-4806-bafd-65d8c35e4ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:26.740717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4585e8-0fd1-4806-bafd-65d8c35e4ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:26.740998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4585e8-0fd1-4806-bafd-65d8c35e4ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:26.741272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4585e8-0fd1-4806-bafd-65d8c35e4ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:26.742927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-be4585e8-0fd1-4806-bafd-65d8c35e4ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:26.849841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4585e8-0fd1-4806-bafd-65d8c35e4ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:26.851288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4585e8-0fd1-4806-bafd-65d8c35e4ac6 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:26.869190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 723/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:28.932294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f292d1-eae3-4892-a678-055c330579c2 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:28.932794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f292d1-eae3-4892-a678-055c330579c2 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:28.932901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f292d1-eae3-4892-a678-055c330579c2 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:28.934242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:28.939534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3928c8-c920-4943-bfac-96b023e76119 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:28.939956 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3928c8-c920-4943-bfac-96b023e76119 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:28.940060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3928c8-c920-4943-bfac-96b023e76119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:29.002686 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-aa3928c8-c920-4943-bfac-96b023e76119 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:29.003906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-aa3928c8-c920-4943-bfac-96b023e76119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:29.011865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:29.100075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:29.136774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f292d1-eae3-4892-a678-055c330579c2 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:29.138861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e1f292d1-eae3-4892-a678-055c330579c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:29.247809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f292d1-eae3-4892-a678-055c330579c2 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:29.302537 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:29.320470 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb52035-b6e4-4599-8b63-a1f4b33b93d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:29.321433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb52035-b6e4-4599-8b63-a1f4b33b93d2 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:29.321717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb52035-b6e4-4599-8b63-a1f4b33b93d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:29.321987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb52035-b6e4-4599-8b63-a1f4b33b93d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:29.323632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-dfb52035-b6e4-4599-8b63-a1f4b33b93d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:29.432302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb52035-b6e4-4599-8b63-a1f4b33b93d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:29.433747 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb52035-b6e4-4599-8b63-a1f4b33b93d2 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:29.452997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 737/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:31.528369 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825cc8b-5d6e-4b92-b584-eac01916cc80 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:31.528902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825cc8b-5d6e-4b92-b584-eac01916cc80 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:31.528902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825cc8b-5d6e-4b92-b584-eac01916cc80 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:31.530298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:31.535907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946111d6-f09d-4ff4-9093-335c5f3e3dcb None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:31.536311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946111d6-f09d-4ff4-9093-335c5f3e3dcb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:31.536453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946111d6-f09d-4ff4-9093-335c5f3e3dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:31.598558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-946111d6-f09d-4ff4-9093-335c5f3e3dcb None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:31.599691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-946111d6-f09d-4ff4-9093-335c5f3e3dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:31.637521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:31.698902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:31.736053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825cc8b-5d6e-4b92-b584-eac01916cc80 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:31.737606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3825cc8b-5d6e-4b92-b584-eac01916cc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:31.841322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825cc8b-5d6e-4b92-b584-eac01916cc80 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:31.879946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:31.915422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32453a5-6032-42a6-94fb-41d3b403d7bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:31.916081 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32453a5-6032-42a6-94fb-41d3b403d7bd service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:31.916366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32453a5-6032-42a6-94fb-41d3b403d7bd service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:31.916647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32453a5-6032-42a6-94fb-41d3b403d7bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:31.918297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b32453a5-6032-42a6-94fb-41d3b403d7bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:32.140862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32453a5-6032-42a6-94fb-41d3b403d7bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:32.142207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32453a5-6032-42a6-94fb-41d3b403d7bd service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:32.160710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 726/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:34.200950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660542f2-35f0-498d-b54a-b63f193f4f12 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:34.201443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660542f2-35f0-498d-b54a-b63f193f4f12 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:34.201584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660542f2-35f0-498d-b54a-b63f193f4f12 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:34.202919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:34.208384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f84601-243d-4cd3-bf28-a87ba1e026b4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:34.208845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f84601-243d-4cd3-bf28-a87ba1e026b4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:34.208945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f84601-243d-4cd3-bf28-a87ba1e026b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:34.271557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-96f84601-243d-4cd3-bf28-a87ba1e026b4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:34.272756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-96f84601-243d-4cd3-bf28-a87ba1e026b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:34.290440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:34.367041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:34.404357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660542f2-35f0-498d-b54a-b63f193f4f12 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:34.405877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-660542f2-35f0-498d-b54a-b63f193f4f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:34.507008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660542f2-35f0-498d-b54a-b63f193f4f12 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:34.577231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:34.585500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa21d66-2ff9-477d-a4aa-d51d5b102a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:34.586160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa21d66-2ff9-477d-a4aa-d51d5b102a7c service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:34.602105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa21d66-2ff9-477d-a4aa-d51d5b102a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:34.602105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa21d66-2ff9-477d-a4aa-d51d5b102a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:34.602105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7fa21d66-2ff9-477d-a4aa-d51d5b102a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:34.711147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa21d66-2ff9-477d-a4aa-d51d5b102a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:34.712687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa21d66-2ff9-477d-a4aa-d51d5b102a7c service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:34.732427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 740/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:36.771502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b086e-bdc7-46ca-b30a-9d7d32b14828 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:36.772088 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b086e-bdc7-46ca-b30a-9d7d32b14828 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:36.772088 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b086e-bdc7-46ca-b30a-9d7d32b14828 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:36.773422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:36.778971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae1b33-88f3-41df-b5fc-4cf8c8692fb5 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:36.779398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae1b33-88f3-41df-b5fc-4cf8c8692fb5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:36.779509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae1b33-88f3-41df-b5fc-4cf8c8692fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:36.841932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-43ae1b33-88f3-41df-b5fc-4cf8c8692fb5 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:36.843148 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-43ae1b33-88f3-41df-b5fc-4cf8c8692fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:36.898988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:36.955912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:36.992885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b086e-bdc7-46ca-b30a-9d7d32b14828 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:36.994709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f69b086e-bdc7-46ca-b30a-9d7d32b14828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:37.097904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b086e-bdc7-46ca-b30a-9d7d32b14828 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:37.145237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:37.170832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8f4fe-4929-4619-9445-5004812b9448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:37.171480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8f4fe-4929-4619-9445-5004812b9448 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:37.171760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8f4fe-4929-4619-9445-5004812b9448 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:37.172032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8f4fe-4929-4619-9445-5004812b9448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:37.173660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-79e8f4fe-4929-4619-9445-5004812b9448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:37.287147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8f4fe-4929-4619-9445-5004812b9448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:37.288591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8f4fe-4929-4619-9445-5004812b9448 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:37.307341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 729/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:46.964431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=22 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:47.305137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=66 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:57.835117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ced0de-a0c9-49d3-84f0-1c985fdd2330 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:57.835657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ced0de-a0c9-49d3-84f0-1c985fdd2330 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:57.835657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ced0de-a0c9-49d3-84f0-1c985fdd2330 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:57.837072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 742/1471] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:57.843491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc88e7c-0140-4720-b740-c223f7149208 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:57.843992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc88e7c-0140-4720-b740-c223f7149208 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:57.844104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc88e7c-0140-4720-b740-c223f7149208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:57.907378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0fc88e7c-0140-4720-b740-c223f7149208 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:57.908832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0fc88e7c-0140-4720-b740-c223f7149208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:58.003133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 730/1472] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:58.086301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a7967-606f-4fbb-9929-5c7d0581a967 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:58.086567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a7967-606f-4fbb-9929-5c7d0581a967 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:58.086850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a7967-606f-4fbb-9929-5c7d0581a967 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:58.088229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 743/1473] 10.222.0.17 () {54 vars in 966 bytes} [Mon Jul 25 05:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:58.093754 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74835976-9a59-42ff-bed1-bbfe1602c7f9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:58.094046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74835976-9a59-42ff-bed1-bbfe1602c7f9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:58.094322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74835976-9a59-42ff-bed1-bbfe1602c7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:58.167240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-74835976-9a59-42ff-bed1-bbfe1602c7f9 None None] MFA Rules not processed for user `6e12361da5f849ee97607ca68ee6ef10`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:58.168442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-74835976-9a59-42ff-bed1-bbfe1602c7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:58.188188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:58.242645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 731/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Mon Jul 25 05:52:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:58.250162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a7967-606f-4fbb-9929-5c7d0581a967 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:58.251679 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-fd3a7967-606f-4fbb-9929-5c7d0581a967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:58.354333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a7967-606f-4fbb-9929-5c7d0581a967 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:58.425353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fdaeb-c2d3-4432-8eac-7238c482788e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e12361da5f849ee97607ca68ee6ef10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:58.425992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fdaeb-c2d3-4432-8eac-7238c482788e service glance] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:58.426273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fdaeb-c2d3-4432-8eac-7238c482788e service glance] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:58.426577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fdaeb-c2d3-4432-8eac-7238c482788e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:58.428178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-242fdaeb-c2d3-4432-8eac-7238c482788e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:58.481769 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:58.538007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fdaeb-c2d3-4432-8eac-7238c482788e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:58.539501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fdaeb-c2d3-4432-8eac-7238c482788e service glance] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:58.557051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 744/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:00.658868 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbba358-10a6-4d77-8ed1-aa93bd16e866 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:00.659437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbba358-10a6-4d77-8ed1-aa93bd16e866 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:00.659437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbba358-10a6-4d77-8ed1-aa93bd16e866 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:00.660790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 732/1476] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:00.666393 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c73d8-6c70-4ed0-8e30-9322c1756eca None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:00.666829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c73d8-6c70-4ed0-8e30-9322c1756eca None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:00.666977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c73d8-6c70-4ed0-8e30-9322c1756eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:00.730865 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d63c73d8-6c70-4ed0-8e30-9322c1756eca None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:00.732634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d63c73d8-6c70-4ed0-8e30-9322c1756eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:00.827942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 745/1477] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:00.851443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbba358-10a6-4d77-8ed1-aa93bd16e866 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:00.852938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3dbba358-10a6-4d77-8ed1-aa93bd16e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:00.975849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbba358-10a6-4d77-8ed1-aa93bd16e866 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:01.022920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:01.066150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5b746-12f5-4f3a-8379-f4fc7458446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:01.066834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5b746-12f5-4f3a-8379-f4fc7458446c admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:01.067114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5b746-12f5-4f3a-8379-f4fc7458446c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:01.067382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5b746-12f5-4f3a-8379-f4fc7458446c admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:01.068102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5b746-12f5-4f3a-8379-f4fc7458446c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:01.069808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WARNING py.warnings [None req-76d5b746-12f5-4f3a-8379-f4fc7458446c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:53:01.069808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: warnings.warn(msg) Jul 25 05:53:01.069808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: Jul 25 05:53:01.070971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5b746-12f5-4f3a-8379-f4fc7458446c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:01.089525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 733/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Jul 25 05:53:00 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:53:03.120887 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbda552-d5f6-47a0-ac47-72d6508127ea None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:03.121460 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbda552-d5f6-47a0-ac47-72d6508127ea None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:03.121557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbda552-d5f6-47a0-ac47-72d6508127ea None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:03.122922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 746/1479] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:03.128208 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a3308-4b22-430e-9404-9402cbe24dd4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:03.128602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a3308-4b22-430e-9404-9402cbe24dd4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:03.128758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a3308-4b22-430e-9404-9402cbe24dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:03.191453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-f79a3308-4b22-430e-9404-9402cbe24dd4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:03.192575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f79a3308-4b22-430e-9404-9402cbe24dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:03.286291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 734/1480] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:03.328778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbda552-d5f6-47a0-ac47-72d6508127ea None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:03.330292 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4fbda552-d5f6-47a0-ac47-72d6508127ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:03.339976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:03.474056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbda552-d5f6-47a0-ac47-72d6508127ea None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:03.551388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:03.583833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd48ba1-f564-46dd-a08f-96574beeeda6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:03.584486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd48ba1-f564-46dd-a08f-96574beeeda6 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:03.584772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd48ba1-f564-46dd-a08f-96574beeeda6 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:03.585049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd48ba1-f564-46dd-a08f-96574beeeda6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:03.586776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2bd48ba1-f564-46dd-a08f-96574beeeda6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:03.677253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd48ba1-f564-46dd-a08f-96574beeeda6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:03.678902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd48ba1-f564-46dd-a08f-96574beeeda6 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:03.697178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 747/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:05.919559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1813e2-34a6-470b-85be-5ecebf6c93cf None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:05.927947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1813e2-34a6-470b-85be-5ecebf6c93cf None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:05.927947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1813e2-34a6-470b-85be-5ecebf6c93cf None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:05.927947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 735/1482] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:53:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:05.928238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68dc2a-c39d-4189-a5ad-7dd97cc12ef5 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:05.928238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68dc2a-c39d-4189-a5ad-7dd97cc12ef5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:05.928238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68dc2a-c39d-4189-a5ad-7dd97cc12ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:05.990887 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-ba68dc2a-c39d-4189-a5ad-7dd97cc12ef5 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:05.992106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ba68dc2a-c39d-4189-a5ad-7dd97cc12ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:06.037095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:06.085963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 748/1483] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:06.122580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1813e2-34a6-470b-85be-5ecebf6c93cf None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:06.124104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0e1813e2-34a6-470b-85be-5ecebf6c93cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:06.147231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:06.227462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1813e2-34a6-470b-85be-5ecebf6c93cf None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:06.301929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a029d-508f-4086-b8bd-062f3ae90b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:06.302613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a029d-508f-4086-b8bd-062f3ae90b44 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:06.302894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a029d-508f-4086-b8bd-062f3ae90b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:06.303176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a029d-508f-4086-b8bd-062f3ae90b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:06.304792 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e77a029d-508f-4086-b8bd-062f3ae90b44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:06.376291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:06.411181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a029d-508f-4086-b8bd-062f3ae90b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:06.412619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a029d-508f-4086-b8bd-062f3ae90b44 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:06.431292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 736/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:08.549838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbe99f-8fe3-40ce-9db4-4f9fb2b60a08 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:08.550383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbe99f-8fe3-40ce-9db4-4f9fb2b60a08 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:08.550383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbe99f-8fe3-40ce-9db4-4f9fb2b60a08 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:08.551749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 749/1485] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:08.556826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21b66d-2c28-40dd-9d5b-44dbe4e151f4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:08.557256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21b66d-2c28-40dd-9d5b-44dbe4e151f4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:08.557369 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21b66d-2c28-40dd-9d5b-44dbe4e151f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:08.621445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3d21b66d-2c28-40dd-9d5b-44dbe4e151f4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:08.622679 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3d21b66d-2c28-40dd-9d5b-44dbe4e151f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:08.715897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 737/1486] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:08.754372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbe99f-8fe3-40ce-9db4-4f9fb2b60a08 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:08.755926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-85fbe99f-8fe3-40ce-9db4-4f9fb2b60a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:08.861477 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbe99f-8fe3-40ce-9db4-4f9fb2b60a08 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:08.932333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:08.934096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a6eba-53c0-4f0c-8511-0f91b05ff790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:08.934769 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a6eba-53c0-4f0c-8511-0f91b05ff790 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:08.935050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a6eba-53c0-4f0c-8511-0f91b05ff790 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:08.935322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a6eba-53c0-4f0c-8511-0f91b05ff790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:08.936949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-609a6eba-53c0-4f0c-8511-0f91b05ff790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:09.041498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a6eba-53c0-4f0c-8511-0f91b05ff790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:09.042970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a6eba-53c0-4f0c-8511-0f91b05ff790 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:09.061277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 750/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:11.171040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79812fbd-f7ab-4f8c-aff5-c5acd0621223 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:11.171564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79812fbd-f7ab-4f8c-aff5-c5acd0621223 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:11.171564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79812fbd-f7ab-4f8c-aff5-c5acd0621223 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:11.172892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 738/1488] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:11.178014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e29d56-dc54-41a2-b08c-de7523dc2f29 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:11.178412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e29d56-dc54-41a2-b08c-de7523dc2f29 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:11.178600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e29d56-dc54-41a2-b08c-de7523dc2f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:11.242658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-39e29d56-dc54-41a2-b08c-de7523dc2f29 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:11.243864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-39e29d56-dc54-41a2-b08c-de7523dc2f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:11.295369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:11.337025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 751/1489] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:11.377808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79812fbd-f7ab-4f8c-aff5-c5acd0621223 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:11.379444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-79812fbd-f7ab-4f8c-aff5-c5acd0621223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:11.414991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:11.482572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79812fbd-f7ab-4f8c-aff5-c5acd0621223 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:11.553850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d3cff-9d0f-4463-8fb8-981c4ac129eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:11.554510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d3cff-9d0f-4463-8fb8-981c4ac129eb service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:11.554793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d3cff-9d0f-4463-8fb8-981c4ac129eb service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:11.555070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d3cff-9d0f-4463-8fb8-981c4ac129eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:11.556673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8c8d3cff-9d0f-4463-8fb8-981c4ac129eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:11.634898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:11.663992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d3cff-9d0f-4463-8fb8-981c4ac129eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:11.665516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d3cff-9d0f-4463-8fb8-981c4ac129eb service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:11.683525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 739/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:14.124929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc84b4-7400-416c-9470-ab88be1212b5 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:14.125522 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc84b4-7400-416c-9470-ab88be1212b5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:14.125522 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc84b4-7400-416c-9470-ab88be1212b5 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:14.126937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 752/1491] 10.222.0.17 () {54 vars in 926 bytes} [Mon Jul 25 05:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:14.132072 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f9a4-7923-45c1-b0ba-346a49b619d9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:14.132518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f9a4-7923-45c1-b0ba-346a49b619d9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:14.132714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f9a4-7923-45c1-b0ba-346a49b619d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:14.196707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-9c49f9a4-7923-45c1-b0ba-346a49b619d9 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:14.197863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9c49f9a4-7923-45c1-b0ba-346a49b619d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:14.294202 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 740/1492] 10.222.0.17 () {58 vars in 1023 bytes} [Mon Jul 25 05:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:14.332718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc84b4-7400-416c-9470-ab88be1212b5 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:14.334261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2bbc84b4-7400-416c-9470-ab88be1212b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:14.435415 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc84b4-7400-416c-9470-ab88be1212b5 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:14.506984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54d222-9619-48ca-a5f4-c74e3e5b781b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:14.507548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54d222-9619-48ca-a5f4-c74e3e5b781b service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:14.507825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54d222-9619-48ca-a5f4-c74e3e5b781b service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:14.508095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54d222-9619-48ca-a5f4-c74e3e5b781b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:14.509702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ea54d222-9619-48ca-a5f4-c74e3e5b781b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:14.519130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:14.615428 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54d222-9619-48ca-a5f4-c74e3e5b781b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:14.616866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54d222-9619-48ca-a5f4-c74e3e5b781b service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:14.634264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 753/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:24.291813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=82 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:24.632019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=50 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:48.829079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a221-6950-4e44-b665-ee1eab026a86 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:48.829696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a221-6950-4e44-b665-ee1eab026a86 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:48.829696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a221-6950-4e44-b665-ee1eab026a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:48.901930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0324a221-6950-4e44-b665-ee1eab026a86 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:54:48.903173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0324a221-6950-4e44-b665-ee1eab026a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:48.997254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 741/1494] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:54:49.005698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a50eb0-8ab7-4181-8ad4-cb8e93cc37cf None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:49.006197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a50eb0-8ab7-4181-8ad4-cb8e93cc37cf None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:49.006307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a50eb0-8ab7-4181-8ad4-cb8e93cc37cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:49.079994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a1a50eb0-8ab7-4181-8ad4-cb8e93cc37cf None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:54:49.081170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a1a50eb0-8ab7-4181-8ad4-cb8e93cc37cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:49.176399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 754/1495] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:54:49.181006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324a221-6950-4e44-b665-ee1eab026a86 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:49.182210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0324a221-6950-4e44-b665-ee1eab026a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:49.193620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324a221-6950-4e44-b665-ee1eab026a86 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:49.254029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:49.289030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:49.289695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:49.289977 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:49.290243 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:49.290987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:54:49.291288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:49.292821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WARNING py.warnings [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:54:49.292821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: warnings.warn(msg) Jul 25 05:54:49.292821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: Jul 25 05:54:49.293720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:49.305938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 742/1496] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:54:49.314336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a50eb0-8ab7-4181-8ad4-cb8e93cc37cf None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:49.323337 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a50eb0-8ab7-4181-8ad4-cb8e93cc37cf None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:49.383448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:49.418511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b388da9-49eb-4d91-a0c3-50950a718b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:49.419177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b388da9-49eb-4d91-a0c3-50950a718b80 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:49.419457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b388da9-49eb-4d91-a0c3-50950a718b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:49.419813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b388da9-49eb-4d91-a0c3-50950a718b80 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:49.421013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b388da9-49eb-4d91-a0c3-50950a718b80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:49.422768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: WARNING py.warnings [None req-6b388da9-49eb-4d91-a0c3-50950a718b80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:54:49.422768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: warnings.warn(msg) Jul 25 05:54:49.422768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: Jul 25 05:54:49.423962 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b388da9-49eb-4d91-a0c3-50950a718b80 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:49.552488 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 755/1497] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:54:49 2022] POST /identity/v3/projects => generated 379 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:54:49.556777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:49.558290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:49.591382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc1c3d-b996-4239-b86c-e52f5f8822f9 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:49.683383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6a0a7-65f7-403d-ad42-c45613767e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:49.684163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6a0a7-65f7-403d-ad42-c45613767e1e admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:49.684283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6a0a7-65f7-403d-ad42-c45613767e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:49.684503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6a0a7-65f7-403d-ad42-c45613767e1e admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:49.685194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6a0a7-65f7-403d-ad42-c45613767e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:49.686236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WARNING py.warnings [None req-61e6a0a7-65f7-403d-ad42-c45613767e1e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:54:49.686236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: warnings.warn(msg) Jul 25 05:54:49.686236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: Jul 25 05:54:49.687514 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6a0a7-65f7-403d-ad42-c45613767e1e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:49.787501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 743/1498] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:54:49 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:54:49.791732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b388da9-49eb-4d91-a0c3-50950a718b80 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:49.793365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6b388da9-49eb-4d91-a0c3-50950a718b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:49.855266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b388da9-49eb-4d91-a0c3-50950a718b80 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:49.937833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:49.947776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c41229-7b50-4858-bef3-8d5e7997dc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:49.948444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c41229-7b50-4858-bef3-8d5e7997dc5f admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:49.952161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c41229-7b50-4858-bef3-8d5e7997dc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:49.952161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c41229-7b50-4858-bef3-8d5e7997dc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:49.952161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c41229-7b50-4858-bef3-8d5e7997dc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:49.952161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: WARNING py.warnings [None req-e5c41229-7b50-4858-bef3-8d5e7997dc5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:54:49.952161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: warnings.warn(msg) Jul 25 05:54:49.952161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: Jul 25 05:54:49.952161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c41229-7b50-4858-bef3-8d5e7997dc5f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:49.964300 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 756/1499] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:54:49.973915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6a0a7-65f7-403d-ad42-c45613767e1e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:49.975466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-61e6a0a7-65f7-403d-ad42-c45613767e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:49.987799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6a0a7-65f7-403d-ad42-c45613767e1e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:50.014387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:50.082420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c49687-6bcf-4e55-91ae-15cde832f1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:50.083121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c49687-6bcf-4e55-91ae-15cde832f1ab admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:50.083402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c49687-6bcf-4e55-91ae-15cde832f1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:50.083678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c49687-6bcf-4e55-91ae-15cde832f1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:50.084400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c49687-6bcf-4e55-91ae-15cde832f1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:50.085405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WARNING py.warnings [None req-e9c49687-6bcf-4e55-91ae-15cde832f1ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:54:50.085405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: warnings.warn(msg) Jul 25 05:54:50.085405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: Jul 25 05:54:50.086285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c49687-6bcf-4e55-91ae-15cde832f1ab admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:50.102827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 744/1500] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:54:50.107171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c41229-7b50-4858-bef3-8d5e7997dc5f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:50.108708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e5c41229-7b50-4858-bef3-8d5e7997dc5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:50.143020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c41229-7b50-4858-bef3-8d5e7997dc5f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:50.241114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8bedf-2947-4019-98b8-755c5e858a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:50.241921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8bedf-2947-4019-98b8-755c5e858a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:50.242197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8bedf-2947-4019-98b8-755c5e858a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:50.242464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8bedf-2947-4019-98b8-755c5e858a12 admin admin] PATH_INFO: `/v3/projects/d8bd22952bd9460d8ef8bfda151a69c8/users/14ae48e8e35e4706acc3addb1821cc2a/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:50.267583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8bedf-2947-4019-98b8-755c5e858a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8bd22952bd9460d8ef8bfda151a69c8, user_id=14ae48e8e35e4706acc3addb1821cc2a, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:50.273080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: WARNING py.warnings [None req-aea8bedf-2947-4019-98b8-755c5e858a12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:54:50.273080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: warnings.warn(msg) Jul 25 05:54:50.273080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: Jul 25 05:54:50.273080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8bedf-2947-4019-98b8-755c5e858a12 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:50.356602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 757/1501] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:54:50 2022] PUT /identity/v3/projects/d8bd22952bd9460d8ef8bfda151a69c8/users/14ae48e8e35e4706acc3addb1821cc2a/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:54:50.373607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a534e2f-0420-4565-b974-315a3a1247a1 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:50.374220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a534e2f-0420-4565-b974-315a3a1247a1 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:50.374643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a534e2f-0420-4565-b974-315a3a1247a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:50.432535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-9a534e2f-0420-4565-b974-315a3a1247a1 None None] MFA Rules not processed for user `14ae48e8e35e4706acc3addb1821cc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:54:50.433170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9a534e2f-0420-4565-b974-315a3a1247a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:50.515391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 745/1502] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:54:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:54:50.528976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5976e7-29b1-4356-9b75-ab294d9bdc0e None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:50.529856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5976e7-29b1-4356-9b75-ab294d9bdc0e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:50.530138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5976e7-29b1-4356-9b75-ab294d9bdc0e None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:50.531629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 758/1503] 10.222.0.17 () {54 vars in 964 bytes} [Mon Jul 25 05:54:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:54:50.537034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae846c-9c0b-4566-99f0-52cf2a668323 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:50.538710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae846c-9c0b-4566-99f0-52cf2a668323 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:50.538710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae846c-9c0b-4566-99f0-52cf2a668323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:50.569947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:50.620317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-26ae846c-9c0b-4566-99f0-52cf2a668323 None None] MFA Rules not processed for user `6640e4c799514429a2fe7850369af4df`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:54:50.621559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-26ae846c-9c0b-4566-99f0-52cf2a668323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:50.708907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 746/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Mon Jul 25 05:54:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:54:50.716802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5976e7-29b1-4356-9b75-ab294d9bdc0e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:50.718300 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9b5976e7-29b1-4356-9b75-ab294d9bdc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:50.772596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:50.827793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5976e7-29b1-4356-9b75-ab294d9bdc0e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:50.899319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bafce-ef2e-481e-8f1c-d130538f4e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:50.899969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bafce-ef2e-481e-8f1c-d130538f4e59 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:50.900247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bafce-ef2e-481e-8f1c-d130538f4e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:50.900516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bafce-ef2e-481e-8f1c-d130538f4e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:50.902130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3a3bafce-ef2e-481e-8f1c-d130538f4e59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:50.997804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bafce-ef2e-481e-8f1c-d130538f4e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:50.999832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bafce-ef2e-481e-8f1c-d130538f4e59 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:51.003286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:51.018319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 759/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:54:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:54:51.061568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330123b-103d-4cb1-a6c8-88a080eddbd2 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:51.061996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330123b-103d-4cb1-a6c8-88a080eddbd2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:51.062107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330123b-103d-4cb1-a6c8-88a080eddbd2 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:51.063408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 747/1506] 10.222.0.17 () {52 vars in 829 bytes} [Mon Jul 25 05:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:54:51.067002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bafce-ef2e-481e-8f1c-d130538f4e59 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:51.068230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3a3bafce-ef2e-481e-8f1c-d130538f4e59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:51.163338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bafce-ef2e-481e-8f1c-d130538f4e59 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:51.254670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:51.255552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:51.255810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:51.256080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] PATH_INFO: `/v3/users/14ae48e8e35e4706acc3addb1821cc2a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:51.267986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ae48e8e35e4706acc3addb1821cc2a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:51.269022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: WARNING py.warnings [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:54:51.269022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: warnings.warn(msg) Jul 25 05:54:51.269022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: Jul 25 05:54:51.270267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:51.280507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:51.469618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ae48e8e35e4706acc3addb1821cc2a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:54:51.481911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ae48e8e35e4706acc3addb1821cc2a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:54:51.560830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ae48e8e35e4706acc3addb1821cc2a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:54:51.561198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ae48e8e35e4706acc3addb1821cc2a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:54:51.561713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ae48e8e35e4706acc3addb1821cc2a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:54:51.584858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 760/1507] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:54:51 2022] DELETE /identity/v3/users/14ae48e8e35e4706acc3addb1821cc2a => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:54:51.589232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395f82c-547f-4982-8632-621972bcceb4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:51.589554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395f82c-547f-4982-8632-621972bcceb4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:51.589835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395f82c-547f-4982-8632-621972bcceb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:51.644352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3395f82c-547f-4982-8632-621972bcceb4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:54:51.645554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3395f82c-547f-4982-8632-621972bcceb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:51.721291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:51.733269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 748/1508] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:54:51.743016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:51.748758 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:51.842985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b00bc-9951-4da8-ab6d-d64cd06da96e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:51.895748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:51.915020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11214d-3cda-4414-97ca-2cf3c531b743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:51.915673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11214d-3cda-4414-97ca-2cf3c531b743 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:51.915958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11214d-3cda-4414-97ca-2cf3c531b743 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:51.916237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11214d-3cda-4414-97ca-2cf3c531b743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:51.917872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6b11214d-3cda-4414-97ca-2cf3c531b743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:52.026992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11214d-3cda-4414-97ca-2cf3c531b743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:52.028480 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11214d-3cda-4414-97ca-2cf3c531b743 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:52.047359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 761/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:54:52.494648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395f82c-547f-4982-8632-621972bcceb4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:52.495904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3395f82c-547f-4982-8632-621972bcceb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:52.508464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395f82c-547f-4982-8632-621972bcceb4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:52.608273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:52.609223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:52.609506 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:52.609782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] PATH_INFO: `/v3/projects/d8bd22952bd9460d8ef8bfda151a69c8` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:52.619941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8bd22952bd9460d8ef8bfda151a69c8)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:52.621163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WARNING py.warnings [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:54:52.621163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: warnings.warn(msg) Jul 25 05:54:52.621163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: Jul 25 05:54:52.622367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:52.733566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8bd22952bd9460d8ef8bfda151a69c8', 'request_id': 'req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:54:52.748807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8bd22952bd9460d8ef8bfda151a69c8', 'request_id': 'req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:54:52.754833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8bd22952bd9460d8ef8bfda151a69c8', 'request_id': 'req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:54:52.842723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] The token cache is being invalidate because project d8bd22952bd9460d8ef8bfda151a69c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:54:52.843011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:54:52.843530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a72a90d7-5f39-4f15-afa6-5ea64b8aa47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:54:52.844884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 749/1510] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:54:52 2022] DELETE /identity/v3/projects/d8bd22952bd9460d8ef8bfda151a69c8 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:55:02.045028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=62 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:02.840225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:56:34.093724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce33801-bf4a-4614-8a2d-5b815816e717 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:34.094325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce33801-bf4a-4614-8a2d-5b815816e717 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:34.094325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce33801-bf4a-4614-8a2d-5b815816e717 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:34.095843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:56:34.100738 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce63bf4-9251-4067-aa8e-ff3a8b1b559c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:34.101182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce63bf4-9251-4067-aa8e-ff3a8b1b559c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:34.101287 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce63bf4-9251-4067-aa8e-ff3a8b1b559c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:34.179325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0ce63bf4-9251-4067-aa8e-ff3a8b1b559c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:56:34.180449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0ce63bf4-9251-4067-aa8e-ff3a8b1b559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:34.282702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:56:34.436182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce33801-bf4a-4614-8a2d-5b815816e717 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:56:34.437228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cce33801-bf4a-4614-8a2d-5b815816e717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:34.554101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce33801-bf4a-4614-8a2d-5b815816e717 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:56:34.559589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5e840-2a7c-45ca-acb8-86958be2f95a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:56:34.561448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5e840-2a7c-45ca-acb8-86958be2f95a service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:34.561448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5e840-2a7c-45ca-acb8-86958be2f95a service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:34.561448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5e840-2a7c-45ca-acb8-86958be2f95a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:34.569554 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5e840-2a7c-45ca-acb8-86958be2f95a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:56:34.571258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5e840-2a7c-45ca-acb8-86958be2f95a service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:56:34.580873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 763/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:56:35.416658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a40c2a-835b-4519-83d5-9077107dc2a7 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:35.417253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a40c2a-835b-4519-83d5-9077107dc2a7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:35.417253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a40c2a-835b-4519-83d5-9077107dc2a7 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:35.418658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:56:35.423356 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120f13-beb6-4fc5-b2e1-ed5cafb3854f None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:35.423772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120f13-beb6-4fc5-b2e1-ed5cafb3854f None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:35.423926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120f13-beb6-4fc5-b2e1-ed5cafb3854f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:35.442324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a7120f13-beb6-4fc5-b2e1-ed5cafb3854f None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:56:35.452214 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a7120f13-beb6-4fc5-b2e1-ed5cafb3854f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:35.452214 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:56:35.603211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a40c2a-835b-4519-83d5-9077107dc2a7 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:56:35.617472 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a40c2a-835b-4519-83d5-9077107dc2a7 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:56:35.622988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72ab44-1538-421e-95f6-e3194082b10f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:56:35.623656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72ab44-1538-421e-95f6-e3194082b10f service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:35.623943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72ab44-1538-421e-95f6-e3194082b10f service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:35.624223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72ab44-1538-421e-95f6-e3194082b10f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:35.632879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72ab44-1538-421e-95f6-e3194082b10f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:56:35.634461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72ab44-1538-421e-95f6-e3194082b10f service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:56:35.644447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 752/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:56:36.303167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70ade6-f2d3-489c-8857-f3e065e48ff5 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:36.303456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70ade6-f2d3-489c-8857-f3e065e48ff5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:36.303738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70ade6-f2d3-489c-8857-f3e065e48ff5 None None] PATH_INFO: `/v3` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:36.305197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 765/1517] 10.222.0.42 () {54 vars in 925 bytes} [Mon Jul 25 05:56:36 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:56:36.311388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec549250-2ef4-427a-b8ff-8e42f83c164b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:36.311677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec549250-2ef4-427a-b8ff-8e42f83c164b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:36.311957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec549250-2ef4-427a-b8ff-8e42f83c164b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:36.368724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ec549250-2ef4-427a-b8ff-8e42f83c164b None None] MFA Rules not processed for user `dbfcf2e915c049f08a9e41c555401a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:56:36.369966 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ec549250-2ef4-427a-b8ff-8e42f83c164b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:36.387015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:56:36.441205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 753/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Jul 25 05:56:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:56:40.399540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e54304-9d1a-4171-bbf2-ae34b669ee8b None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:40.400130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e54304-9d1a-4171-bbf2-ae34b669ee8b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:40.400130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e54304-9d1a-4171-bbf2-ae34b669ee8b None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:40.401529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 766/1519] 10.222.0.17 () {54 vars in 961 bytes} [Mon Jul 25 05:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:56:40.406337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e7929-a4ba-4e0e-99fc-91986ac1f7e9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:40.406791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e7929-a4ba-4e0e-99fc-91986ac1f7e9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:40.407027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e7929-a4ba-4e0e-99fc-91986ac1f7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:40.425359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-7f5e7929-a4ba-4e0e-99fc-91986ac1f7e9 None None] MFA Rules not processed for user `dbfcf2e915c049f08a9e41c555401a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:56:40.426582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7f5e7929-a4ba-4e0e-99fc-91986ac1f7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:40.435056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 754/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Jul 25 05:56:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:56:40.442278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e54304-9d1a-4171-bbf2-ae34b669ee8b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:56:40.456063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e54304-9d1a-4171-bbf2-ae34b669ee8b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:56:40.460996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa1c8a-fe3a-4a1a-9115-e23e2f790ae0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:56:40.461622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1c8a-fe3a-4a1a-9115-e23e2f790ae0 service placement] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:40.461905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1c8a-fe3a-4a1a-9115-e23e2f790ae0 service placement] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:40.462183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1c8a-fe3a-4a1a-9115-e23e2f790ae0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:40.471050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa1c8a-fe3a-4a1a-9115-e23e2f790ae0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:56:40.472625 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa1c8a-fe3a-4a1a-9115-e23e2f790ae0 service placement] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:56:40.482305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 767/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:56:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:56:46.841646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1b092-a213-46b3-8296-dc9f375fe9ea None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:46.842245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1b092-a213-46b3-8296-dc9f375fe9ea None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:46.842245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1b092-a213-46b3-8296-dc9f375fe9ea None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:46.843651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:56:46.848531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166faf25-183f-479d-9c4c-1e7b1ea0596a None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:46.848970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166faf25-183f-479d-9c4c-1e7b1ea0596a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:46.849070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166faf25-183f-479d-9c4c-1e7b1ea0596a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:46.866818 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-166faf25-183f-479d-9c4c-1e7b1ea0596a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:56:46.868019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-166faf25-183f-479d-9c4c-1e7b1ea0596a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:46.878346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:56:47.032093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1b092-a213-46b3-8296-dc9f375fe9ea None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:56:47.051136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1b092-a213-46b3-8296-dc9f375fe9ea None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:56:47.056880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a034b8-d60e-441d-983e-451083b7bd38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:56:47.057515 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a034b8-d60e-441d-983e-451083b7bd38 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:47.057794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a034b8-d60e-441d-983e-451083b7bd38 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:47.058064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a034b8-d60e-441d-983e-451083b7bd38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:47.066893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a034b8-d60e-441d-983e-451083b7bd38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:56:47.068497 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a034b8-d60e-441d-983e-451083b7bd38 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:56:47.078684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 756/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:56:47.904651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cf220-7517-4b36-afae-c9fc55609aae None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:47.905232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cf220-7517-4b36-afae-c9fc55609aae None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:47.905232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cf220-7517-4b36-afae-c9fc55609aae None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:47.906631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:56:47.912008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cec5c-022d-43b6-91b7-953a9022cd4b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:47.912428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cec5c-022d-43b6-91b7-953a9022cd4b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:47.912558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cec5c-022d-43b6-91b7-953a9022cd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:47.930701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2a6cec5c-022d-43b6-91b7-953a9022cd4b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:56:47.931903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2a6cec5c-022d-43b6-91b7-953a9022cd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:47.940386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:56:48.090833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cf220-7517-4b36-afae-c9fc55609aae None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:56:48.104994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cf220-7517-4b36-afae-c9fc55609aae None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:56:48.110950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5031520-708f-4891-86c2-5507188b5a5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:56:48.111615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5031520-708f-4891-86c2-5507188b5a5a service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:48.111897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5031520-708f-4891-86c2-5507188b5a5a service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:48.112173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5031520-708f-4891-86c2-5507188b5a5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:48.120221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5031520-708f-4891-86c2-5507188b5a5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:56:48.121808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5031520-708f-4891-86c2-5507188b5a5a service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:56:48.131771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 770/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:56:52.301281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84adb2-e755-4798-8c00-eda2992cc736 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:52.301801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84adb2-e755-4798-8c00-eda2992cc736 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:52.301801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84adb2-e755-4798-8c00-eda2992cc736 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:52.303184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:56:52.308606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5aaa8-41a0-4e86-8d12-92ac00676dca None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:52.309029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5aaa8-41a0-4e86-8d12-92ac00676dca None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:52.309164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5aaa8-41a0-4e86-8d12-92ac00676dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:52.327581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-6ad5aaa8-41a0-4e86-8d12-92ac00676dca None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:56:52.328829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6ad5aaa8-41a0-4e86-8d12-92ac00676dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:52.337435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:56:52.415969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84adb2-e755-4798-8c00-eda2992cc736 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:56:52.417685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6c84adb2-e755-4798-8c00-eda2992cc736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:52.506506 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84adb2-e755-4798-8c00-eda2992cc736 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:56:52.510092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0fb8f-1b10-41fe-96c8-f661afc8cfd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:56:52.510771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0fb8f-1b10-41fe-96c8-f661afc8cfd7 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:52.511053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0fb8f-1b10-41fe-96c8-f661afc8cfd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:52.511327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0fb8f-1b10-41fe-96c8-f661afc8cfd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:52.522211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0fb8f-1b10-41fe-96c8-f661afc8cfd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:56:52.523683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0fb8f-1b10-41fe-96c8-f661afc8cfd7 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:56:52.534078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 759/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:56:54.569940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce0fd-8758-4581-9b23-bde2de814615 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:54.570616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce0fd-8758-4581-9b23-bde2de814615 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:54.570616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce0fd-8758-4581-9b23-bde2de814615 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:54.571911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:56:54.576983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124849c-3170-4d1e-8989-e6be4d66ee8f None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:54.577392 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124849c-3170-4d1e-8989-e6be4d66ee8f None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:54.577540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124849c-3170-4d1e-8989-e6be4d66ee8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:54.595605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-f124849c-3170-4d1e-8989-e6be4d66ee8f None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:56:54.596805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f124849c-3170-4d1e-8989-e6be4d66ee8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:54.604786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:56:54.642801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ce0fd-8758-4581-9b23-bde2de814615 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:56:54.653366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ce0fd-8758-4581-9b23-bde2de814615 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:56:54.656857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627bace-2f52-4b00-a7be-8023dfdaf91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:56:54.657510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627bace-2f52-4b00-a7be-8023dfdaf91e service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:54.657792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627bace-2f52-4b00-a7be-8023dfdaf91e service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:54.658067 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627bace-2f52-4b00-a7be-8023dfdaf91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:54.667133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627bace-2f52-4b00-a7be-8023dfdaf91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:56:54.668558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627bace-2f52-4b00-a7be-8023dfdaf91e service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:56:54.678315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 773/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:56:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:56:58.294393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c22b19-d294-42d4-9f15-cd4178e83ac2 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:58.294954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c22b19-d294-42d4-9f15-cd4178e83ac2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:58.294954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c22b19-d294-42d4-9f15-cd4178e83ac2 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:58.296306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:56:58.301501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ee4a1-afa3-4040-aaf2-9beb9c151d8d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:58.301926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ee4a1-afa3-4040-aaf2-9beb9c151d8d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:58.302049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ee4a1-afa3-4040-aaf2-9beb9c151d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:58.319966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-682ee4a1-afa3-4040-aaf2-9beb9c151d8d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:56:58.321144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-682ee4a1-afa3-4040-aaf2-9beb9c151d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:58.328888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:56:58.634967 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c22b19-d294-42d4-9f15-cd4178e83ac2 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:56:58.636638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f8c22b19-d294-42d4-9f15-cd4178e83ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:56:58.701696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:56:58.715108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c22b19-d294-42d4-9f15-cd4178e83ac2 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:56:58.719569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556dd7e-0b9f-4180-83ca-7bce702d42d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:56:58.719569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556dd7e-0b9f-4180-83ca-7bce702d42d3 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:56:58.719832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556dd7e-0b9f-4180-83ca-7bce702d42d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:56:58.719937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556dd7e-0b9f-4180-83ca-7bce702d42d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:56:58.729127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556dd7e-0b9f-4180-83ca-7bce702d42d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:56:58.730626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556dd7e-0b9f-4180-83ca-7bce702d42d3 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:56:58.741222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 762/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:57:00.798191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cb9e5-b4f4-4cb7-9c9a-3c7390cadb44 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:00.798827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cb9e5-b4f4-4cb7-9c9a-3c7390cadb44 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:00.798827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cb9e5-b4f4-4cb7-9c9a-3c7390cadb44 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:00.800166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:57:00.805391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bcfff-a026-428e-8b81-c0638e461a2d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:00.805828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bcfff-a026-428e-8b81-c0638e461a2d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:00.805941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bcfff-a026-428e-8b81-c0638e461a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:00.824331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-502bcfff-a026-428e-8b81-c0638e461a2d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:00.825534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-502bcfff-a026-428e-8b81-c0638e461a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:00.833932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:57:01.134364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cb9e5-b4f4-4cb7-9c9a-3c7390cadb44 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:57:01.136080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1d3cb9e5-b4f4-4cb7-9c9a-3c7390cadb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:01.149325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cb9e5-b4f4-4cb7-9c9a-3c7390cadb44 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:57:01.152867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750d217-6098-4f24-ab75-e066d1efd414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:57:01.153502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d217-6098-4f24-ab75-e066d1efd414 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:01.153781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d217-6098-4f24-ab75-e066d1efd414 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:01.154058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d217-6098-4f24-ab75-e066d1efd414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:01.163685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750d217-6098-4f24-ab75-e066d1efd414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:57:01.165103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750d217-6098-4f24-ab75-e066d1efd414 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:57:01.174417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 776/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:57:10.819682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=11 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:57:11.170243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:31.067993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69917cb-ffb0-4261-a452-cb9432d141d7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.069179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69917cb-ffb0-4261-a452-cb9432d141d7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.069990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69917cb-ffb0-4261-a452-cb9432d141d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.092361 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e69917cb-ffb0-4261-a452-cb9432d141d7 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.093566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e69917cb-ffb0-4261-a452-cb9432d141d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.112012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 764/1540] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.121369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536bda6-e7bc-4433-ac2a-96dd70375876 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.122377 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536bda6-e7bc-4433-ac2a-96dd70375876 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.122862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536bda6-e7bc-4433-ac2a-96dd70375876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.134194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8ee6-a00f-4173-8214-71c1b642c778 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.134499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8ee6-a00f-4173-8214-71c1b642c778 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.134769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e8ee6-a00f-4173-8214-71c1b642c778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.158044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-bc6e8ee6-a00f-4173-8214-71c1b642c778 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.159706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-c536bda6-e7bc-4433-ac2a-96dd70375876 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.159841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bc6e8ee6-a00f-4173-8214-71c1b642c778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.161083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c536bda6-e7bc-4433-ac2a-96dd70375876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.174901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 765/1541] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:58:31.181365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 777/1542] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.183805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175868-a00f-4f74-a8d3-008bf5bf3191 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.184093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175868-a00f-4f74-a8d3-008bf5bf3191 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.184360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175868-a00f-4f74-a8d3-008bf5bf3191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.185949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536bda6-e7bc-4433-ac2a-96dd70375876 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.201177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536bda6-e7bc-4433-ac2a-96dd70375876 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.209358 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e0175868-a00f-4f74-a8d3-008bf5bf3191 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.214898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e0175868-a00f-4f74-a8d3-008bf5bf3191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.222510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe3ba4-d107-4df4-afb3-a812c26a769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.223228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe3ba4-d107-4df4-afb3-a812c26a769f admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.223522 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe3ba4-d107-4df4-afb3-a812c26a769f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.223808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe3ba4-d107-4df4-afb3-a812c26a769f admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.224594 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe3ba4-d107-4df4-afb3-a812c26a769f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:58:31.224967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe3ba4-d107-4df4-afb3-a812c26a769f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.225982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: WARNING py.warnings [None req-2cbe3ba4-d107-4df4-afb3-a812c26a769f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:58:31.225982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: warnings.warn(msg) Jul 25 05:58:31.225982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: Jul 25 05:58:31.226953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe3ba4-d107-4df4-afb3-a812c26a769f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.227894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 766/1543] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.230698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0175868-a00f-4f74-a8d3-008bf5bf3191 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.245773 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0175868-a00f-4f74-a8d3-008bf5bf3191 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.246008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 778/1544] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:58:31.251438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388864a1-0587-4b73-9ee7-d0a1999374a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.252141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388864a1-0587-4b73-9ee7-d0a1999374a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.252421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388864a1-0587-4b73-9ee7-d0a1999374a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.252700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388864a1-0587-4b73-9ee7-d0a1999374a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.253680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388864a1-0587-4b73-9ee7-d0a1999374a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:58:31.253994 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388864a1-0587-4b73-9ee7-d0a1999374a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.255463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe3ba4-d107-4df4-afb3-a812c26a769f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.255679 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388864a1-0587-4b73-9ee7-d0a1999374a2 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.271560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe3ba4-d107-4df4-afb3-a812c26a769f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.282658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 767/1545] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:58:31.285739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388864a1-0587-4b73-9ee7-d0a1999374a2 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.295272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c16295-36a4-4c34-a114-96116dfc4e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.295968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c16295-36a4-4c34-a114-96116dfc4e65 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.296256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c16295-36a4-4c34-a114-96116dfc4e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.296536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c16295-36a4-4c34-a114-96116dfc4e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.296830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388864a1-0587-4b73-9ee7-d0a1999374a2 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.297248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c16295-36a4-4c34-a114-96116dfc4e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.299420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c16295-36a4-4c34-a114-96116dfc4e65 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.305953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76ffdd-8353-488c-9e27-9b0c238be87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.306659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76ffdd-8353-488c-9e27-9b0c238be87a admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.306944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76ffdd-8353-488c-9e27-9b0c238be87a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.307469 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76ffdd-8353-488c-9e27-9b0c238be87a admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.308260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76ffdd-8353-488c-9e27-9b0c238be87a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.309277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WARNING py.warnings [None req-ef76ffdd-8353-488c-9e27-9b0c238be87a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:58:31.309277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: warnings.warn(msg) Jul 25 05:58:31.309277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: Jul 25 05:58:31.310462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76ffdd-8353-488c-9e27-9b0c238be87a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.447317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 779/1546] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/projects => generated 391 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:31.448559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ce45e-5e19-4c8f-a319-8796562628be None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.448852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ce45e-5e19-4c8f-a319-8796562628be None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.449134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ce45e-5e19-4c8f-a319-8796562628be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.472934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e76ce45e-5e19-4c8f-a319-8796562628be None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.474441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e76ce45e-5e19-4c8f-a319-8796562628be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.504415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 768/1547] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/projects => generated 371 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:58:31.505629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76ffdd-8353-488c-9e27-9b0c238be87a admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.565860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 780/1548] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.566982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9a02-8516-41b9-838b-e8da6d0a7461 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.567275 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9a02-8516-41b9-838b-e8da6d0a7461 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.567556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c9a02-8516-41b9-838b-e8da6d0a7461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.589313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-230c9a02-8516-41b9-838b-e8da6d0a7461 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.590561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-230c9a02-8516-41b9-838b-e8da6d0a7461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.592719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76ffdd-8353-488c-9e27-9b0c238be87a admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.597692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1adb95-aea5-4c2f-a790-1827c933add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.598493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1adb95-aea5-4c2f-a790-1827c933add0 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.598898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1adb95-aea5-4c2f-a790-1827c933add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.599278 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1adb95-aea5-4c2f-a790-1827c933add0 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.605461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 781/1549] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.605640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1adb95-aea5-4c2f-a790-1827c933add0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.605744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5792c-319f-49ac-bef6-044127979a50 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.605744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5792c-319f-49ac-bef6-044127979a50 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.605744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5792c-319f-49ac-bef6-044127979a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.606064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1adb95-aea5-4c2f-a790-1827c933add0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.623194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-3fe5792c-319f-49ac-bef6-044127979a50 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.626550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3fe5792c-319f-49ac-bef6-044127979a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.633262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 782/1550] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.634106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5792c-319f-49ac-bef6-044127979a50 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.643794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5792c-319f-49ac-bef6-044127979a50 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.648477 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e26f0a-2e07-4bc6-8e8f-1db3d0e0c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.649143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e26f0a-2e07-4bc6-8e8f-1db3d0e0c0da admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.649426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e26f0a-2e07-4bc6-8e8f-1db3d0e0c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.649700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e26f0a-2e07-4bc6-8e8f-1db3d0e0c0da admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.650387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e26f0a-2e07-4bc6-8e8f-1db3d0e0c0da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.651455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: WARNING py.warnings [None req-71e26f0a-2e07-4bc6-8e8f-1db3d0e0c0da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:58:31.651455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: warnings.warn(msg) Jul 25 05:58:31.651455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: Jul 25 05:58:31.652638 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e26f0a-2e07-4bc6-8e8f-1db3d0e0c0da admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.713521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 769/1551] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:31.713521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59590592-68b3-460e-acc1-73adb9db9450 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.713521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59590592-68b3-460e-acc1-73adb9db9450 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.713521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59590592-68b3-460e-acc1-73adb9db9450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.730322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-59590592-68b3-460e-acc1-73adb9db9450 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.731597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-59590592-68b3-460e-acc1-73adb9db9450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.738465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 783/1552] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:31.738465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423812b6-0676-480d-b847-ef0e99769384 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.738465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423812b6-0676-480d-b847-ef0e99769384 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.738465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423812b6-0676-480d-b847-ef0e99769384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.740627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 770/1553] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.741865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0024fa-75b9-4c1c-83f5-3e60d1201ae4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.742152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0024fa-75b9-4c1c-83f5-3e60d1201ae4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.742433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0024fa-75b9-4c1c-83f5-3e60d1201ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.756355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-423812b6-0676-480d-b847-ef0e99769384 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.757575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-423812b6-0676-480d-b847-ef0e99769384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.760831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-5c0024fa-75b9-4c1c-83f5-3e60d1201ae4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.761997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5c0024fa-75b9-4c1c-83f5-3e60d1201ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.765734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 784/1554] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.767063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcbe8c-b275-4e6f-bd0f-072239c0ff7c None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.767271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcbe8c-b275-4e6f-bd0f-072239c0ff7c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.767551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcbe8c-b275-4e6f-bd0f-072239c0ff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.771193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 771/1555] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.772563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930af34-f0d4-4518-84b4-368909221191 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.778943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930af34-f0d4-4518-84b4-368909221191 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.778943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930af34-f0d4-4518-84b4-368909221191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.785274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-34bcbe8c-b275-4e6f-bd0f-072239c0ff7c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.786517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-34bcbe8c-b275-4e6f-bd0f-072239c0ff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.792405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8930af34-f0d4-4518-84b4-368909221191 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.793583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8930af34-f0d4-4518-84b4-368909221191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.794905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 785/1556] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.796115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f53e92-c4f5-47f8-936d-ff5d0f1d2c34 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.796410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f53e92-c4f5-47f8-936d-ff5d0f1d2c34 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.796694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f53e92-c4f5-47f8-936d-ff5d0f1d2c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.804944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 772/1557] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.807462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930af34-f0d4-4518-84b4-368909221191 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.815263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-71f53e92-c4f5-47f8-936d-ff5d0f1d2c34 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.816459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-71f53e92-c4f5-47f8-936d-ff5d0f1d2c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.819004 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930af34-f0d4-4518-84b4-368909221191 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.823593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393946b2-d9d3-4456-a030-e09db82c2ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.824246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393946b2-d9d3-4456-a030-e09db82c2ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.824533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393946b2-d9d3-4456-a030-e09db82c2ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.824796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 786/1558] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.824890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393946b2-d9d3-4456-a030-e09db82c2ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.825512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393946b2-d9d3-4456-a030-e09db82c2ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.825634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f53e92-c4f5-47f8-936d-ff5d0f1d2c34 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.827137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393946b2-d9d3-4456-a030-e09db82c2ac9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.835926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f53e92-c4f5-47f8-936d-ff5d0f1d2c34 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.840534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 773/1559] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:31.840534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87524b5d-4fa5-4e5f-a798-a71b6d3f7a38 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.840534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87524b5d-4fa5-4e5f-a798-a71b6d3f7a38 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.840534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87524b5d-4fa5-4e5f-a798-a71b6d3f7a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.843394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6cf25-682d-421f-a632-09ccfa94dfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.844043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6cf25-682d-421f-a632-09ccfa94dfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.844321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6cf25-682d-421f-a632-09ccfa94dfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.844592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6cf25-682d-421f-a632-09ccfa94dfb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.845317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6cf25-682d-421f-a632-09ccfa94dfb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.846983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6cf25-682d-421f-a632-09ccfa94dfb6 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.859741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 787/1560] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:31.859921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-87524b5d-4fa5-4e5f-a798-a71b6d3f7a38 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.860589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a20f-2ebd-4735-b501-5131565f420b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.860589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a20f-2ebd-4735-b501-5131565f420b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.860790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-87524b5d-4fa5-4e5f-a798-a71b6d3f7a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.860899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a20f-2ebd-4735-b501-5131565f420b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.867801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 774/1561] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.868590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87524b5d-4fa5-4e5f-a798-a71b6d3f7a38 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.877259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-fa98a20f-2ebd-4735-b501-5131565f420b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.878435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-fa98a20f-2ebd-4735-b501-5131565f420b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.878583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87524b5d-4fa5-4e5f-a798-a71b6d3f7a38 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.883197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651a176-affb-4398-9502-f3420f30e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.883841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651a176-affb-4398-9502-f3420f30e47c admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.884124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651a176-affb-4398-9502-f3420f30e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.884401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651a176-affb-4398-9502-f3420f30e47c admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.885116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651a176-affb-4398-9502-f3420f30e47c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:58:31.885423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651a176-affb-4398-9502-f3420f30e47c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.886259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 788/1562] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.887036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a20f-2ebd-4735-b501-5131565f420b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.887821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651a176-affb-4398-9502-f3420f30e47c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.896031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a20f-2ebd-4735-b501-5131565f420b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.900051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 775/1563] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:58:31.907365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a94a73-0a2c-4403-b55d-5589c61f270b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.907476 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209d0fa-e834-46b5-ac3e-f16476cb0969 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.907476 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209d0fa-e834-46b5-ac3e-f16476cb0969 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.907632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a94a73-0a2c-4403-b55d-5589c61f270b admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.907733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209d0fa-e834-46b5-ac3e-f16476cb0969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.907821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a94a73-0a2c-4403-b55d-5589c61f270b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.907821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a94a73-0a2c-4403-b55d-5589c61f270b admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.907821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a94a73-0a2c-4403-b55d-5589c61f270b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:58:31.907821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a94a73-0a2c-4403-b55d-5589c61f270b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.907821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a94a73-0a2c-4403-b55d-5589c61f270b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.918263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 789/1564] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:58:31.918937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a94a73-0a2c-4403-b55d-5589c61f270b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.923931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c209d0fa-e834-46b5-ac3e-f16476cb0969 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:31.925856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c209d0fa-e834-46b5-ac3e-f16476cb0969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:31.928854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a94a73-0a2c-4403-b55d-5589c61f270b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.933416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823d63a-1dab-4c12-b402-53d0e29908bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.933995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 776/1565] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:31.937668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823d63a-1dab-4c12-b402-53d0e29908bb admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.937668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823d63a-1dab-4c12-b402-53d0e29908bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.937668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823d63a-1dab-4c12-b402-53d0e29908bb admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.937918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209d0fa-e834-46b5-ac3e-f16476cb0969 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.938007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823d63a-1dab-4c12-b402-53d0e29908bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:58:31.938007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823d63a-1dab-4c12-b402-53d0e29908bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.938007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823d63a-1dab-4c12-b402-53d0e29908bb admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.944606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209d0fa-e834-46b5-ac3e-f16476cb0969 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.949903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 790/1566] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:58:31.950718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823d63a-1dab-4c12-b402-53d0e29908bb admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.950852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07d4a7-1f72-419e-87eb-3980600b9b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.951475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07d4a7-1f72-419e-87eb-3980600b9b0d admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.951690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07d4a7-1f72-419e-87eb-3980600b9b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.951956 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07d4a7-1f72-419e-87eb-3980600b9b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.952677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07d4a7-1f72-419e-87eb-3980600b9b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.954324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07d4a7-1f72-419e-87eb-3980600b9b0d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.962043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823d63a-1dab-4c12-b402-53d0e29908bb admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.964895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 777/1567] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:31.965507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07d4a7-1f72-419e-87eb-3980600b9b0d admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.970424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136780c-3fcc-4107-b0d0-0d6f63baae2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.971104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136780c-3fcc-4107-b0d0-0d6f63baae2c admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.971480 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136780c-3fcc-4107-b0d0-0d6f63baae2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.971760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136780c-3fcc-4107-b0d0-0d6f63baae2c admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.972483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136780c-3fcc-4107-b0d0-0d6f63baae2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.974068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136780c-3fcc-4107-b0d0-0d6f63baae2c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.977608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07d4a7-1f72-419e-87eb-3980600b9b0d admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.982381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2dcd6-2616-4210-a1eb-8dfe9d389d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:31.983048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2dcd6-2616-4210-a1eb-8dfe9d389d49 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:31.983333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2dcd6-2616-4210-a1eb-8dfe9d389d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:31.983614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2dcd6-2616-4210-a1eb-8dfe9d389d49 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:31.987804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 791/1568] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:31.987954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2dcd6-2616-4210-a1eb-8dfe9d389d49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:58:31.987954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2dcd6-2616-4210-a1eb-8dfe9d389d49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:31.988128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136780c-3fcc-4107-b0d0-0d6f63baae2c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:31.988214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2dcd6-2616-4210-a1eb-8dfe9d389d49 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:31.992996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:31.996818 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136780c-3fcc-4107-b0d0-0d6f63baae2c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:31.999938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 778/1569] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:58:32.000767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2dcd6-2616-4210-a1eb-8dfe9d389d49 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.001768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612ea47-3167-4358-8701-9720d7277d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.002429 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612ea47-3167-4358-8701-9720d7277d92 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.002751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612ea47-3167-4358-8701-9720d7277d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.003162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612ea47-3167-4358-8701-9720d7277d92 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.003898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612ea47-3167-4358-8701-9720d7277d92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:58:32.004207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612ea47-3167-4358-8701-9720d7277d92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.005804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612ea47-3167-4358-8701-9720d7277d92 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.008756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:32.012888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2dcd6-2616-4210-a1eb-8dfe9d389d49 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.017250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5a493-d1c2-4d13-8edf-1c0961056994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.018123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5a493-d1c2-4d13-8edf-1c0961056994 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.018240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 792/1570] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:58:32.018324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5a493-d1c2-4d13-8edf-1c0961056994 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.018506 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5a493-d1c2-4d13-8edf-1c0961056994 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.018960 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612ea47-3167-4358-8701-9720d7277d92 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.019245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5a493-d1c2-4d13-8edf-1c0961056994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.021066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5a493-d1c2-4d13-8edf-1c0961056994 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.028422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612ea47-3167-4358-8701-9720d7277d92 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.033605 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9ba13-6edd-43ab-8a1c-a242beda4802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.034250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9ba13-6edd-43ab-8a1c-a242beda4802 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.034612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9ba13-6edd-43ab-8a1c-a242beda4802 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.034898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9ba13-6edd-43ab-8a1c-a242beda4802 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.035658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9ba13-6edd-43ab-8a1c-a242beda4802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.037886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9ba13-6edd-43ab-8a1c-a242beda4802 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.128937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 779/1572] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:31 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:32.129512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 793/1572] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/projects => generated 403 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:32.129897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5a493-d1c2-4d13-8edf-1c0961056994 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.133218 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9ba13-6edd-43ab-8a1c-a242beda4802 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.196305 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5a493-d1c2-4d13-8edf-1c0961056994 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.199795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9ba13-6edd-43ab-8a1c-a242beda4802 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.201177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24934cd-d628-4d5d-af80-c70ba17e5a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.201845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24934cd-d628-4d5d-af80-c70ba17e5a00 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.202133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24934cd-d628-4d5d-af80-c70ba17e5a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.202408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24934cd-d628-4d5d-af80-c70ba17e5a00 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.203190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24934cd-d628-4d5d-af80-c70ba17e5a00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:58:32.203499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24934cd-d628-4d5d-af80-c70ba17e5a00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.204468 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006c229-9b30-4175-8aa1-1b59a2a4438b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.205092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24934cd-d628-4d5d-af80-c70ba17e5a00 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.205200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c229-9b30-4175-8aa1-1b59a2a4438b admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.205402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c229-9b30-4175-8aa1-1b59a2a4438b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.205691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c229-9b30-4175-8aa1-1b59a2a4438b admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.206387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006c229-9b30-4175-8aa1-1b59a2a4438b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.208316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006c229-9b30-4175-8aa1-1b59a2a4438b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.217641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 780/1573] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:58:32.218787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24934cd-d628-4d5d-af80-c70ba17e5a00 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.229031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24934cd-d628-4d5d-af80-c70ba17e5a00 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.233752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591a394-5269-4573-8446-7fbfa9185d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.234722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591a394-5269-4573-8446-7fbfa9185d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.235008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591a394-5269-4573-8446-7fbfa9185d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.235829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591a394-5269-4573-8446-7fbfa9185d39 admin admin] PATH_INFO: `/v3/projects/d30518907b9f4939b1e8666eba12186e/users/13e5d648af294c7cbc86ccd362914728/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.250490 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591a394-5269-4573-8446-7fbfa9185d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30518907b9f4939b1e8666eba12186e, user_id=13e5d648af294c7cbc86ccd362914728, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.251554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WARNING py.warnings [None req-b591a394-5269-4573-8446-7fbfa9185d39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:58:32.251554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: warnings.warn(msg) Jul 25 05:58:32.251554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: Jul 25 05:58:32.253503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591a394-5269-4573-8446-7fbfa9185d39 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.289611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 794/1574] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:32.290413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006c229-9b30-4175-8aa1-1b59a2a4438b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.350718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006c229-9b30-4175-8aa1-1b59a2a4438b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.351043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 781/1575] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:32 2022] PUT /identity/v3/projects/d30518907b9f4939b1e8666eba12186e/users/13e5d648af294c7cbc86ccd362914728/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:32.351983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591a394-5269-4573-8446-7fbfa9185d39 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.407948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1adc8-262c-4e85-b274-94556b01db1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.408844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1adc8-262c-4e85-b274-94556b01db1d admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.409116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1adc8-262c-4e85-b274-94556b01db1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.409392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1adc8-262c-4e85-b274-94556b01db1d admin admin] PATH_INFO: `/v3/projects/06767cd5344d4d4bac5d4903001d0e28/users/677031e79ac04fe385988f107ef67a77/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.413494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:32.414066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591a394-5269-4573-8446-7fbfa9185d39 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.419309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf603eb-7852-4b12-9759-30041e377ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.420012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf603eb-7852-4b12-9759-30041e377ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.420246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf603eb-7852-4b12-9759-30041e377ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.420524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf603eb-7852-4b12-9759-30041e377ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.421210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf603eb-7852-4b12-9759-30041e377ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.423689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf603eb-7852-4b12-9759-30041e377ca8 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.425088 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1adc8-262c-4e85-b274-94556b01db1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=06767cd5344d4d4bac5d4903001d0e28, user_id=677031e79ac04fe385988f107ef67a77, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.427799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1adc8-262c-4e85-b274-94556b01db1d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.504039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 795/1576] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:32 2022] PUT /identity/v3/projects/06767cd5344d4d4bac5d4903001d0e28/users/677031e79ac04fe385988f107ef67a77/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:32.504756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1adc8-262c-4e85-b274-94556b01db1d admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.530182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 782/1577] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:32.531357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf603eb-7852-4b12-9759-30041e377ca8 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.568079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1adc8-262c-4e85-b274-94556b01db1d admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.573512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:32.593378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf603eb-7852-4b12-9759-30041e377ca8 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.598436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c975ac-817d-4fc4-b328-fb31b536bac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.599140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c975ac-817d-4fc4-b328-fb31b536bac4 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.600171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c975ac-817d-4fc4-b328-fb31b536bac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.600171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c975ac-817d-4fc4-b328-fb31b536bac4 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.600422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c975ac-817d-4fc4-b328-fb31b536bac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.602340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c975ac-817d-4fc4-b328-fb31b536bac4 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.624746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e7c05-5243-4fe2-a9f8-e0779bd22f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.625420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e7c05-5243-4fe2-a9f8-e0779bd22f88 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.625759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e7c05-5243-4fe2-a9f8-e0779bd22f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.626557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e7c05-5243-4fe2-a9f8-e0779bd22f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.627255 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e7c05-5243-4fe2-a9f8-e0779bd22f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.629226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e7c05-5243-4fe2-a9f8-e0779bd22f88 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.667843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 783/1578] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:32.667843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c975ac-817d-4fc4-b328-fb31b536bac4 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.675888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c975ac-817d-4fc4-b328-fb31b536bac4 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.680467 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8a708-af4b-4824-8457-d9d03512971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.681110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8a708-af4b-4824-8457-d9d03512971e admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.681396 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8a708-af4b-4824-8457-d9d03512971e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.681678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8a708-af4b-4824-8457-d9d03512971e admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.682355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8a708-af4b-4824-8457-d9d03512971e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.684292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8a708-af4b-4824-8457-d9d03512971e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.740884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 796/1579] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/projects => generated 387 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:32.741624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e7c05-5243-4fe2-a9f8-e0779bd22f88 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.764586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 784/1580] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:32.764586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8a708-af4b-4824-8457-d9d03512971e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.771795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8a708-af4b-4824-8457-d9d03512971e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.804758 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e7c05-5243-4fe2-a9f8-e0779bd22f88 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.809479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b435850-7663-49f2-9ca6-0a67abd9db41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.810201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b435850-7663-49f2-9ca6-0a67abd9db41 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.810562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b435850-7663-49f2-9ca6-0a67abd9db41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.810792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b435850-7663-49f2-9ca6-0a67abd9db41 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.811489 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b435850-7663-49f2-9ca6-0a67abd9db41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.813411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b435850-7663-49f2-9ca6-0a67abd9db41 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.828975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3bc28-f41a-45e9-83ed-2267dc5bb29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.829729 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3bc28-f41a-45e9-83ed-2267dc5bb29c admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.830016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3bc28-f41a-45e9-83ed-2267dc5bb29c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.830293 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3bc28-f41a-45e9-83ed-2267dc5bb29c admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.831010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3bc28-f41a-45e9-83ed-2267dc5bb29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.832923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3bc28-f41a-45e9-83ed-2267dc5bb29c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:32.888409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 797/1581] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:32.889707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c36e59-4a32-44ed-92b2-04e101d89baa None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.890001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c36e59-4a32-44ed-92b2-04e101d89baa None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.890282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c36e59-4a32-44ed-92b2-04e101d89baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.895938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:32.907991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-68c36e59-4a32-44ed-92b2-04e101d89baa None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:32.909362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-68c36e59-4a32-44ed-92b2-04e101d89baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:32.910319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 785/1582] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:32.912046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e872-9eac-45fc-acf4-0b4829afc4ad None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.913456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e872-9eac-45fc-acf4-0b4829afc4ad None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.913456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e872-9eac-45fc-acf4-0b4829afc4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.938818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-93d9e872-9eac-45fc-acf4-0b4829afc4ad None None] MFA Rules not processed for user `677031e79ac04fe385988f107ef67a77`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:32.940022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-93d9e872-9eac-45fc-acf4-0b4829afc4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:32.968809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 798/1583] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:32.970168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c36e59-4a32-44ed-92b2-04e101d89baa None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:32.981529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c36e59-4a32-44ed-92b2-04e101d89baa None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:32.987154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c28ad6-1c9d-4f10-a155-984aaff8ebad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:32.987822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c28ad6-1c9d-4f10-a155-984aaff8ebad admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:32.988111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c28ad6-1c9d-4f10-a155-984aaff8ebad admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:32.988399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c28ad6-1c9d-4f10-a155-984aaff8ebad admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:32.989316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c28ad6-1c9d-4f10-a155-984aaff8ebad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:32.991477 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c28ad6-1c9d-4f10-a155-984aaff8ebad admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.011016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 786/1584] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:33.012526 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e872-9eac-45fc-acf4-0b4829afc4ad None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.021723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e872-9eac-45fc-acf4-0b4829afc4ad None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.026182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d4c4e-ba1f-435b-9a69-d79237a725e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.026855 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d4c4e-ba1f-435b-9a69-d79237a725e6 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.027136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d4c4e-ba1f-435b-9a69-d79237a725e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.027415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d4c4e-ba1f-435b-9a69-d79237a725e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.028137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d4c4e-ba1f-435b-9a69-d79237a725e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.029702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d4c4e-ba1f-435b-9a69-d79237a725e6 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.040360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 787/1585] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.041108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d4c4e-ba1f-435b-9a69-d79237a725e6 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.050791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d4c4e-ba1f-435b-9a69-d79237a725e6 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.055311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9832e-e6cd-4d29-b207-c12bbeb1cd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.055923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9832e-e6cd-4d29-b207-c12bbeb1cd7c admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.056204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9832e-e6cd-4d29-b207-c12bbeb1cd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.056482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9832e-e6cd-4d29-b207-c12bbeb1cd7c admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.057144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9832e-e6cd-4d29-b207-c12bbeb1cd7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.059017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9832e-e6cd-4d29-b207-c12bbeb1cd7c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.070693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 799/1586] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:32 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:33.071731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c28ad6-1c9d-4f10-a155-984aaff8ebad admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.081258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c28ad6-1c9d-4f10-a155-984aaff8ebad admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.086053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3cd66-dc44-436e-aeaf-25a41d008693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.086738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3cd66-dc44-436e-aeaf-25a41d008693 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.087024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3cd66-dc44-436e-aeaf-25a41d008693 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.087302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3cd66-dc44-436e-aeaf-25a41d008693 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.088026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3cd66-dc44-436e-aeaf-25a41d008693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.089647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3cd66-dc44-436e-aeaf-25a41d008693 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.099982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 800/1587] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.103213 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3cd66-dc44-436e-aeaf-25a41d008693 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.109933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3cd66-dc44-436e-aeaf-25a41d008693 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.115042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34235a-b5ec-4994-ad15-88da3e26b1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.115646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34235a-b5ec-4994-ad15-88da3e26b1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.115931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34235a-b5ec-4994-ad15-88da3e26b1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.116210 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34235a-b5ec-4994-ad15-88da3e26b1d2 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.116904 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34235a-b5ec-4994-ad15-88da3e26b1d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.118760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34235a-b5ec-4994-ad15-88da3e26b1d2 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.151508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:33.156636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 788/1588] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:33 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:33.158059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9832e-e6cd-4d29-b207-c12bbeb1cd7c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.167623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9832e-e6cd-4d29-b207-c12bbeb1cd7c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.172193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb0022-8153-4fb4-a54b-fa752c73205d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.172831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0022-8153-4fb4-a54b-fa752c73205d admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.173120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0022-8153-4fb4-a54b-fa752c73205d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.173401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb0022-8153-4fb4-a54b-fa752c73205d admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.174114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb0022-8153-4fb4-a54b-fa752c73205d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.175743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb0022-8153-4fb4-a54b-fa752c73205d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.186049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 789/1589] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.186850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb0022-8153-4fb4-a54b-fa752c73205d admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.228519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 801/1590] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:33 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:33.228519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34235a-b5ec-4994-ad15-88da3e26b1d2 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.236159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34235a-b5ec-4994-ad15-88da3e26b1d2 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.240843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1250df1-aa91-440e-8365-80ed01eb032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.241493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1250df1-aa91-440e-8365-80ed01eb032a admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.241776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1250df1-aa91-440e-8365-80ed01eb032a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.241920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb0022-8153-4fb4-a54b-fa752c73205d admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.242050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1250df1-aa91-440e-8365-80ed01eb032a admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.242784 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1250df1-aa91-440e-8365-80ed01eb032a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.244764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1250df1-aa91-440e-8365-80ed01eb032a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.246460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc6073-80e2-4c0c-a11a-4c75cd994720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.247086 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc6073-80e2-4c0c-a11a-4c75cd994720 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.247367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc6073-80e2-4c0c-a11a-4c75cd994720 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.247640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc6073-80e2-4c0c-a11a-4c75cd994720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.256932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc6073-80e2-4c0c-a11a-4c75cd994720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.261487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc6073-80e2-4c0c-a11a-4c75cd994720 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.268427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 790/1591] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:33.269212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc6073-80e2-4c0c-a11a-4c75cd994720 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.279466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc6073-80e2-4c0c-a11a-4c75cd994720 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.284777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32181c-319a-4903-a569-fd0e4931ccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.285430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32181c-319a-4903-a569-fd0e4931ccab admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.285712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32181c-319a-4903-a569-fd0e4931ccab admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.285989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32181c-319a-4903-a569-fd0e4931ccab admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.286988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32181c-319a-4903-a569-fd0e4931ccab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.288748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32181c-319a-4903-a569-fd0e4931ccab admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.299065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 791/1592] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.299963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32181c-319a-4903-a569-fd0e4931ccab admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.309298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32181c-319a-4903-a569-fd0e4931ccab admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.314194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae7d83-d25c-47ae-a68c-715ae673c969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.314858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae7d83-d25c-47ae-a68c-715ae673c969 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.315143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae7d83-d25c-47ae-a68c-715ae673c969 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.315419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae7d83-d25c-47ae-a68c-715ae673c969 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.316108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae7d83-d25c-47ae-a68c-715ae673c969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.317635 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae7d83-d25c-47ae-a68c-715ae673c969 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.327218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 792/1593] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.327976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae7d83-d25c-47ae-a68c-715ae673c969 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.337653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae7d83-d25c-47ae-a68c-715ae673c969 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.342373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd15271-cb0f-44dd-afcf-92cc611962f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.343007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15271-cb0f-44dd-afcf-92cc611962f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.343295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15271-cb0f-44dd-afcf-92cc611962f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.343576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15271-cb0f-44dd-afcf-92cc611962f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.344271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd15271-cb0f-44dd-afcf-92cc611962f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.346370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd15271-cb0f-44dd-afcf-92cc611962f0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.357666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 793/1594] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.358446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd15271-cb0f-44dd-afcf-92cc611962f0 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.369651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 802/1595] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:33 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:33.369651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1250df1-aa91-440e-8365-80ed01eb032a admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.370041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd15271-cb0f-44dd-afcf-92cc611962f0 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.432576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1250df1-aa91-440e-8365-80ed01eb032a admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.432937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3d8df-8abd-45f6-b447-f5b9a1eafbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.433623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3d8df-8abd-45f6-b447-f5b9a1eafbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.433901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3d8df-8abd-45f6-b447-f5b9a1eafbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.434180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3d8df-8abd-45f6-b447-f5b9a1eafbd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.434940 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3d8df-8abd-45f6-b447-f5b9a1eafbd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.436540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3d8df-8abd-45f6-b447-f5b9a1eafbd8 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.437536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524035f-717d-4bb8-8643-9cac04c7f88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.438245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524035f-717d-4bb8-8643-9cac04c7f88d admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.438567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524035f-717d-4bb8-8643-9cac04c7f88d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.438859 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524035f-717d-4bb8-8643-9cac04c7f88d admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.439602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524035f-717d-4bb8-8643-9cac04c7f88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.441214 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524035f-717d-4bb8-8643-9cac04c7f88d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.447778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 794/1596] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.448757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3d8df-8abd-45f6-b447-f5b9a1eafbd8 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.458220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 803/1597] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.458220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524035f-717d-4bb8-8643-9cac04c7f88d admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.463308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3d8df-8abd-45f6-b447-f5b9a1eafbd8 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.466634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524035f-717d-4bb8-8643-9cac04c7f88d admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.468525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9aa89-1bb4-443e-8289-dd25a84e7975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.469200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9aa89-1bb4-443e-8289-dd25a84e7975 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.470050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9aa89-1bb4-443e-8289-dd25a84e7975 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.470336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9aa89-1bb4-443e-8289-dd25a84e7975 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.471103 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9aa89-1bb4-443e-8289-dd25a84e7975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.478743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f52f7a-0ffc-4230-93d4-7ec807827b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.478893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9aa89-1bb4-443e-8289-dd25a84e7975 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.479000 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f52f7a-0ffc-4230-93d4-7ec807827b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.479000 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f52f7a-0ffc-4230-93d4-7ec807827b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.479000 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f52f7a-0ffc-4230-93d4-7ec807827b06 admin admin] PATH_INFO: `/v3/projects/bb20b5f0088f4a1c84f17ddb49d851a7/users/3bc10b641e34490a83bc587262f4c61a/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.482954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 795/1598] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.484141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9aa89-1bb4-443e-8289-dd25a84e7975 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.492650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f52f7a-0ffc-4230-93d4-7ec807827b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb20b5f0088f4a1c84f17ddb49d851a7, user_id=3bc10b641e34490a83bc587262f4c61a, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.493674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9aa89-1bb4-443e-8289-dd25a84e7975 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.495897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f52f7a-0ffc-4230-93d4-7ec807827b06 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.498325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ad0ae-f962-43ff-8c8e-9465a9051fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.499018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ad0ae-f962-43ff-8c8e-9465a9051fed admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.499294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ad0ae-f962-43ff-8c8e-9465a9051fed admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.499573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ad0ae-f962-43ff-8c8e-9465a9051fed admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.500284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ad0ae-f962-43ff-8c8e-9465a9051fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.502388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ad0ae-f962-43ff-8c8e-9465a9051fed admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.512276 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 796/1599] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.513045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ad0ae-f962-43ff-8c8e-9465a9051fed admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.522875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ad0ae-f962-43ff-8c8e-9465a9051fed admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.527498 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a83cc9-a421-40ab-be1c-5e51259bcd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.528279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a83cc9-a421-40ab-be1c-5e51259bcd81 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.528568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a83cc9-a421-40ab-be1c-5e51259bcd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.528847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a83cc9-a421-40ab-be1c-5e51259bcd81 admin admin] PATH_INFO: `/v3/projects/73471a51fe474dc18527058f0d0c899d/users/2f35c7969e534c61aa45efff90cc56bb/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.543838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a83cc9-a421-40ab-be1c-5e51259bcd81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73471a51fe474dc18527058f0d0c899d, user_id=2f35c7969e534c61aa45efff90cc56bb, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.546486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a83cc9-a421-40ab-be1c-5e51259bcd81 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.617304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 804/1600] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:33 2022] PUT /identity/v3/projects/bb20b5f0088f4a1c84f17ddb49d851a7/users/3bc10b641e34490a83bc587262f4c61a/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:33.617304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f52f7a-0ffc-4230-93d4-7ec807827b06 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.627032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:33.674260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 797/1601] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:33 2022] PUT /identity/v3/projects/73471a51fe474dc18527058f0d0c899d/users/2f35c7969e534c61aa45efff90cc56bb/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:33.675644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a83cc9-a421-40ab-be1c-5e51259bcd81 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.686108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f52f7a-0ffc-4230-93d4-7ec807827b06 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.689051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:33.743542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ddb56-38a5-43d0-bafc-88fcce7d10fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.744327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ddb56-38a5-43d0-bafc-88fcce7d10fc admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.744613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ddb56-38a5-43d0-bafc-88fcce7d10fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.744889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ddb56-38a5-43d0-bafc-88fcce7d10fc admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.745587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ddb56-38a5-43d0-bafc-88fcce7d10fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.746085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a83cc9-a421-40ab-be1c-5e51259bcd81 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.747711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ddb56-38a5-43d0-bafc-88fcce7d10fc admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.751654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bb948-041d-4e23-b0c5-fe73d87ba798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.752334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bb948-041d-4e23-b0c5-fe73d87ba798 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.752621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bb948-041d-4e23-b0c5-fe73d87ba798 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.752900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bb948-041d-4e23-b0c5-fe73d87ba798 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.753628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bb948-041d-4e23-b0c5-fe73d87ba798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.764513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bb948-041d-4e23-b0c5-fe73d87ba798 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.771025 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 798/1602] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:33.772304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bb948-041d-4e23-b0c5-fe73d87ba798 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.782167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bb948-041d-4e23-b0c5-fe73d87ba798 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.787438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2f28c-5e40-40f4-b007-32f9e6c464b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.788305 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2f28c-5e40-40f4-b007-32f9e6c464b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.788585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2f28c-5e40-40f4-b007-32f9e6c464b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.792895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2f28c-5e40-40f4-b007-32f9e6c464b4 admin admin] PATH_INFO: `/v3/projects/801185ee50e945d1a4c1f7bf75694bf5/users/dcec6f4073664b04a33e6e04fb29eb37/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.803680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2f28c-5e40-40f4-b007-32f9e6c464b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801185ee50e945d1a4c1f7bf75694bf5, user_id=dcec6f4073664b04a33e6e04fb29eb37, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.806516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2f28c-5e40-40f4-b007-32f9e6c464b4 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.900016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 805/1603] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:33 2022] POST /identity/v3/users => generated 292 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:33.900862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ddb56-38a5-43d0-bafc-88fcce7d10fc admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.901932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 799/1604] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:33 2022] PUT /identity/v3/projects/801185ee50e945d1a4c1f7bf75694bf5/users/dcec6f4073664b04a33e6e04fb29eb37/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:33.902864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2f28c-5e40-40f4-b007-32f9e6c464b4 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:33.925438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ddb56-38a5-43d0-bafc-88fcce7d10fc admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.981673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2f28c-5e40-40f4-b007-32f9e6c464b4 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:33.982366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cad6b-a2cd-4954-95e8-c7cc780dade5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.983094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cad6b-a2cd-4954-95e8-c7cc780dade5 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.983372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cad6b-a2cd-4954-95e8-c7cc780dade5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.983652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cad6b-a2cd-4954-95e8-c7cc780dade5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:33.984410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cad6b-a2cd-4954-95e8-c7cc780dade5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:33.986568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cad6b-a2cd-4954-95e8-c7cc780dade5 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:33.987321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07369cf6-706a-46fb-bd6a-d8d0f3ee5db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:33.988793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369cf6-706a-46fb-bd6a-d8d0f3ee5db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:33.989095 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369cf6-706a-46fb-bd6a-d8d0f3ee5db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:33.989380 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07369cf6-706a-46fb-bd6a-d8d0f3ee5db7 admin admin] PATH_INFO: `/v3/projects/15678135086b4a5c83eefece6cb8f79a/users/dc8b3ecc32c14e31b55156586129a40c/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.005144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07369cf6-706a-46fb-bd6a-d8d0f3ee5db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15678135086b4a5c83eefece6cb8f79a, user_id=dc8b3ecc32c14e31b55156586129a40c, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:34.006916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 806/1605] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:34.008272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07369cf6-706a-46fb-bd6a-d8d0f3ee5db7 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:34.008389 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638f6df-4bd2-44b7-8f9e-164809949dd5 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.008596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638f6df-4bd2-44b7-8f9e-164809949dd5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.009002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638f6df-4bd2-44b7-8f9e-164809949dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.027901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e638f6df-4bd2-44b7-8f9e-164809949dd5 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:34.029167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e638f6df-4bd2-44b7-8f9e-164809949dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:34.037868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 807/1606] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:34.039104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10048c40-2cdd-4b47-9837-2ec51648f466 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.039388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10048c40-2cdd-4b47-9837-2ec51648f466 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.039658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10048c40-2cdd-4b47-9837-2ec51648f466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.058702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-10048c40-2cdd-4b47-9837-2ec51648f466 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:34.059909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-10048c40-2cdd-4b47-9837-2ec51648f466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:34.068667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 808/1607] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:34.069598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10048c40-2cdd-4b47-9837-2ec51648f466 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:34.079708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10048c40-2cdd-4b47-9837-2ec51648f466 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:34.084491 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a69aa-f919-4f59-a4d0-ac216810c56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:34.085410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a69aa-f919-4f59-a4d0-ac216810c56e admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.085686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a69aa-f919-4f59-a4d0-ac216810c56e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.085999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a69aa-f919-4f59-a4d0-ac216810c56e admin admin] PATH_INFO: `/v3/projects/d45136a1cdbf4e93b9cf71d33443220f/users/7f269cd38f734209ba0496cf70469608/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.094258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 800/1608] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:33 2022] PUT /identity/v3/projects/15678135086b4a5c83eefece6cb8f79a/users/dc8b3ecc32c14e31b55156586129a40c/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:34.095564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07369cf6-706a-46fb-bd6a-d8d0f3ee5db7 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:34.104442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a69aa-f919-4f59-a4d0-ac216810c56e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45136a1cdbf4e93b9cf71d33443220f, user_id=7f269cd38f734209ba0496cf70469608, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:34.107479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a69aa-f919-4f59-a4d0-ac216810c56e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:34.165632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07369cf6-706a-46fb-bd6a-d8d0f3ee5db7 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:34.168933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 809/1609] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:34 2022] PUT /identity/v3/projects/d45136a1cdbf4e93b9cf71d33443220f/users/7f269cd38f734209ba0496cf70469608/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:34.170529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a671311-c7c5-459a-bc46-91009cd9860d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.170857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a671311-c7c5-459a-bc46-91009cd9860d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.171204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a671311-c7c5-459a-bc46-91009cd9860d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.178114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:34.178748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:34.199972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9a671311-c7c5-459a-bc46-91009cd9860d None None] MFA Rules not processed for user `dcec6f4073664b04a33e6e04fb29eb37`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:34.201223 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9a671311-c7c5-459a-bc46-91009cd9860d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:34.227440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43d95-ca49-4c01-900f-2930a5d1f645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:34.228128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43d95-ca49-4c01-900f-2930a5d1f645 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.228409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43d95-ca49-4c01-900f-2930a5d1f645 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.228684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe43d95-ca49-4c01-900f-2930a5d1f645 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.229410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe43d95-ca49-4c01-900f-2930a5d1f645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:34.231032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe43d95-ca49-4c01-900f-2930a5d1f645 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:34.245279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 801/1610] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:34.245279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43d95-ca49-4c01-900f-2930a5d1f645 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:34.253399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe43d95-ca49-4c01-900f-2930a5d1f645 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:34.258428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223384f-bf08-4c64-a5e3-575364858ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:34.259350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223384f-bf08-4c64-a5e3-575364858ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.259630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223384f-bf08-4c64-a5e3-575364858ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.259910 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223384f-bf08-4c64-a5e3-575364858ac5 admin admin] PATH_INFO: `/v3/projects/488e87a8233e4e69a07cd8efba37cad7/users/6dc49525e3394668b4f1610c4badc144/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.275580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223384f-bf08-4c64-a5e3-575364858ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488e87a8233e4e69a07cd8efba37cad7, user_id=6dc49525e3394668b4f1610c4badc144, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:34.278946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223384f-bf08-4c64-a5e3-575364858ac5 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:34.288753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 810/1611] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:34.288753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a671311-c7c5-459a-bc46-91009cd9860d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:34.351974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a671311-c7c5-459a-bc46-91009cd9860d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:34.355666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ca664-6b5c-4cad-8270-67637d949607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:34.356318 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ca664-6b5c-4cad-8270-67637d949607 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.356596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ca664-6b5c-4cad-8270-67637d949607 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.356866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ca664-6b5c-4cad-8270-67637d949607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.363024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 802/1612] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:34 2022] PUT /identity/v3/projects/488e87a8233e4e69a07cd8efba37cad7/users/6dc49525e3394668b4f1610c4badc144/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:34.367111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ca664-6b5c-4cad-8270-67637d949607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:34.367765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223384f-bf08-4c64-a5e3-575364858ac5 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:34.368555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ca664-6b5c-4cad-8270-67637d949607 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:34.378435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 811/1613] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:34.379727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b515-a9a4-47e1-97ea-196be896058b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.380011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b515-a9a4-47e1-97ea-196be896058b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.380284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b515-a9a4-47e1-97ea-196be896058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.405456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d3c5b515-a9a4-47e1-97ea-196be896058b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:34.406787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d3c5b515-a9a4-47e1-97ea-196be896058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:34.435597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223384f-bf08-4c64-a5e3-575364858ac5 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:34.439400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1d171-7b87-453f-b829-635f76b29d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:34.440090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1d171-7b87-453f-b829-635f76b29d59 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.440376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1d171-7b87-453f-b829-635f76b29d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.440792 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1d171-7b87-453f-b829-635f76b29d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.466127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 812/1614] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:34.467445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941047-66f7-4f4e-b96d-889acb123eba None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.467731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941047-66f7-4f4e-b96d-889acb123eba None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.468004 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941047-66f7-4f4e-b96d-889acb123eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.495925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4e941047-66f7-4f4e-b96d-889acb123eba None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:34.497224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4e941047-66f7-4f4e-b96d-889acb123eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:34.507839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 813/1615] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:34.515131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e941047-66f7-4f4e-b96d-889acb123eba None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:34.520950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e941047-66f7-4f4e-b96d-889acb123eba None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:34.527982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ada57-9235-41e6-82a0-22630a01d752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:34.528898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ada57-9235-41e6-82a0-22630a01d752 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.529246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ada57-9235-41e6-82a0-22630a01d752 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.529583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ada57-9235-41e6-82a0-22630a01d752 admin admin] PATH_INFO: `/v3/projects/70173cb4a9a645e7985642fde941450d/users/92d119d820ba4182a1eee4da178a07eb/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.549011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc1d171-7b87-453f-b829-635f76b29d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:34.549348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ada57-9235-41e6-82a0-22630a01d752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70173cb4a9a645e7985642fde941450d, user_id=92d119d820ba4182a1eee4da178a07eb, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:34.552026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ada57-9235-41e6-82a0-22630a01d752 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:34.552357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc1d171-7b87-453f-b829-635f76b29d59 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:34.568784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 803/1616] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:34.572787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547fa57-10ca-4b87-9b4c-48b097d74dfc None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.572787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547fa57-10ca-4b87-9b4c-48b097d74dfc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.572787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547fa57-10ca-4b87-9b4c-48b097d74dfc None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.573790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 804/1617] 10.222.0.17 () {54 vars in 962 bytes} [Mon Jul 25 05:58:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:58:34.575296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc87e1-cb24-4d37-98d7-23374026eca7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.575732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc87e1-cb24-4d37-98d7-23374026eca7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.576232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc87e1-cb24-4d37-98d7-23374026eca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.607901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0edc87e1-cb24-4d37-98d7-23374026eca7 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:34.607901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0edc87e1-cb24-4d37-98d7-23374026eca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:34.619841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 805/1618] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:34.623412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc87e1-cb24-4d37-98d7-23374026eca7 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:34.625237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0edc87e1-cb24-4d37-98d7-23374026eca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:34.640312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc87e1-cb24-4d37-98d7-23374026eca7 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:34.644556 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3b297-84e3-44c5-a4d6-3baeaa208ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:34.645214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3b297-84e3-44c5-a4d6-3baeaa208ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.645499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3b297-84e3-44c5-a4d6-3baeaa208ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.645778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3b297-84e3-44c5-a4d6-3baeaa208ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.655678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3b297-84e3-44c5-a4d6-3baeaa208ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:34.658076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3b297-84e3-44c5-a4d6-3baeaa208ca9 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:34.665539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 814/1619] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:34 2022] PUT /identity/v3/projects/70173cb4a9a645e7985642fde941450d/users/92d119d820ba4182a1eee4da178a07eb/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:34.665539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ada57-9235-41e6-82a0-22630a01d752 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:34.669503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:34.673511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 806/1620] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:34.675187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95bff4-7dd7-418a-89a2-33d37276ae48 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.675187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95bff4-7dd7-418a-89a2-33d37276ae48 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.675423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95bff4-7dd7-418a-89a2-33d37276ae48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.730567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:34.747890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2b95bff4-7dd7-418a-89a2-33d37276ae48 None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:34.749267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2b95bff4-7dd7-418a-89a2-33d37276ae48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:34.754544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ada57-9235-41e6-82a0-22630a01d752 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:34.767630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47638d86-b650-49cb-999b-b5c3c4e0f5a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:34.775144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47638d86-b650-49cb-999b-b5c3c4e0f5a0 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.776355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47638d86-b650-49cb-999b-b5c3c4e0f5a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.776653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47638d86-b650-49cb-999b-b5c3c4e0f5a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.853662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 807/1621] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Jul 25 05:58:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:34.858892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95bff4-7dd7-418a-89a2-33d37276ae48 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:34.867011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95bff4-7dd7-418a-89a2-33d37276ae48 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:34.879343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e912ff-1db6-4fd0-a794-2190ea933dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:34.882727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e912ff-1db6-4fd0-a794-2190ea933dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.882727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e912ff-1db6-4fd0-a794-2190ea933dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.883459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e912ff-1db6-4fd0-a794-2190ea933dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.923492 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47638d86-b650-49cb-999b-b5c3c4e0f5a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:34.925674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47638d86-b650-49cb-999b-b5c3c4e0f5a0 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:34.938668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 815/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:34.939912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47638d86-b650-49cb-999b-b5c3c4e0f5a0 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:34.955681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47638d86-b650-49cb-999b-b5c3c4e0f5a0 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:34.967509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16912836-e75c-44a0-857c-811d831e7ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:34.970070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16912836-e75c-44a0-857c-811d831e7ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:34.972154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16912836-e75c-44a0-857c-811d831e7ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:34.973906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16912836-e75c-44a0-857c-811d831e7ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:34.974797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16912836-e75c-44a0-857c-811d831e7ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:34.980261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16912836-e75c-44a0-857c-811d831e7ba9 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:34.994519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 816/1623] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:34.995451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16912836-e75c-44a0-857c-811d831e7ba9 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:35.001847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e912ff-1db6-4fd0-a794-2190ea933dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:35.003486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e912ff-1db6-4fd0-a794-2190ea933dd8 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:35.008164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16912836-e75c-44a0-857c-811d831e7ba9 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:35.013527 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 808/1624] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:35.013697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e39c1-cae2-4ee3-8d89-a1f991928c94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:35.014310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e39c1-cae2-4ee3-8d89-a1f991928c94 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:35.014451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e912ff-1db6-4fd0-a794-2190ea933dd8 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:35.014660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e39c1-cae2-4ee3-8d89-a1f991928c94 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:35.014987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e39c1-cae2-4ee3-8d89-a1f991928c94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:35.028206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e912ff-1db6-4fd0-a794-2190ea933dd8 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:35.036309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210cb9b-9c8f-4ee9-97f5-2772bdb8c904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:35.037570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210cb9b-9c8f-4ee9-97f5-2772bdb8c904 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:35.037893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210cb9b-9c8f-4ee9-97f5-2772bdb8c904 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:35.038182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210cb9b-9c8f-4ee9-97f5-2772bdb8c904 admin admin] PATH_INFO: `/v3/projects/70173cb4a9a645e7985642fde941450d/users/92d119d820ba4182a1eee4da178a07eb/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:35.040813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210cb9b-9c8f-4ee9-97f5-2772bdb8c904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70173cb4a9a645e7985642fde941450d, user_id=92d119d820ba4182a1eee4da178a07eb, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:35.051288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210cb9b-9c8f-4ee9-97f5-2772bdb8c904 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:35.059266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 809/1625] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:35 2022] PUT /identity/v3/projects/70173cb4a9a645e7985642fde941450d/users/92d119d820ba4182a1eee4da178a07eb/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:35.072059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bb5f2-1ebc-4f95-b56a-a5c2db6a67dd None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:35.072354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bb5f2-1ebc-4f95-b56a-a5c2db6a67dd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:35.072640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bb5f2-1ebc-4f95-b56a-a5c2db6a67dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:35.133364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-898bb5f2-1ebc-4f95-b56a-a5c2db6a67dd None None] MFA Rules not processed for user `92d119d820ba4182a1eee4da178a07eb`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:35.134897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-898bb5f2-1ebc-4f95-b56a-a5c2db6a67dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:35.136764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e39c1-cae2-4ee3-8d89-a1f991928c94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:35.138528 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e39c1-cae2-4ee3-8d89-a1f991928c94 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:35.161588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 817/1626] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:35.228788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:35.240771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 810/1627] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:35.251977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e39c1-cae2-4ee3-8d89-a1f991928c94 service nova] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:35.361646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:35.382442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e39c1-cae2-4ee3-8d89-a1f991928c94 service nova] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:35.465245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c57437-4daf-4824-bf4a-552b08aa74db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:35.467885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57437-4daf-4824-bf4a-552b08aa74db service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:35.470465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57437-4daf-4824-bf4a-552b08aa74db service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:35.470940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57437-4daf-4824-bf4a-552b08aa74db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:35.491722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c57437-4daf-4824-bf4a-552b08aa74db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:35.491722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c57437-4daf-4824-bf4a-552b08aa74db service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:35.512531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35543420-992e-4399-bc49-585a24eba643 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:35.512912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35543420-992e-4399-bc49-585a24eba643 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:35.513089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35543420-992e-4399-bc49-585a24eba643 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:35.515072 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 811/1628] 10.222.0.17 () {54 vars in 912 bytes} [Mon Jul 25 05:58:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:58:35.523093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c8858-a51e-4c45-a624-fc42333df229 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:35.523093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c8858-a51e-4c45-a624-fc42333df229 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:35.525972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c8858-a51e-4c45-a624-fc42333df229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:35.545107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 818/1629] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:35 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:58:35.563674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-550c8858-a51e-4c45-a624-fc42333df229 None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:35.566114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-550c8858-a51e-4c45-a624-fc42333df229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:35.717571 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 812/1630] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 05:58:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:58:35.733704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c57437-4daf-4824-bf4a-552b08aa74db service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:35.747275 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c57437-4daf-4824-bf4a-552b08aa74db service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:35.751742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0830bc-7adf-445e-aaf3-0b8ad41276e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:35.752838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0830bc-7adf-445e-aaf3-0b8ad41276e5 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:35.753195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0830bc-7adf-445e-aaf3-0b8ad41276e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:35.753583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0830bc-7adf-445e-aaf3-0b8ad41276e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:35.767141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0830bc-7adf-445e-aaf3-0b8ad41276e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:35.768897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0830bc-7adf-445e-aaf3-0b8ad41276e5 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:35.782442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 819/1631] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:58:35.903481 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c8858-a51e-4c45-a624-fc42333df229 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:35.926601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c8858-a51e-4c45-a624-fc42333df229 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:35.940724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3fea3-b4b8-46c5-a84a-152f96cd7d36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:35.941412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3fea3-b4b8-46c5-a84a-152f96cd7d36 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:35.944493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3fea3-b4b8-46c5-a84a-152f96cd7d36 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:35.944493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3fea3-b4b8-46c5-a84a-152f96cd7d36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:36.033412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3fea3-b4b8-46c5-a84a-152f96cd7d36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:36.046805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3fea3-b4b8-46c5-a84a-152f96cd7d36 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:36.073872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 813/1632] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:37.186713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184a122-5b5f-4200-b45b-c2dcdd5d1f95 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:37.186713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184a122-5b5f-4200-b45b-c2dcdd5d1f95 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:37.186713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184a122-5b5f-4200-b45b-c2dcdd5d1f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:37.271282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-2184a122-5b5f-4200-b45b-c2dcdd5d1f95 None None] MFA Rules not processed for user `2f35c7969e534c61aa45efff90cc56bb`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:37.274116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2184a122-5b5f-4200-b45b-c2dcdd5d1f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:37.467659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 820/1633] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:37.471708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3fea3-b4b8-46c5-a84a-152f96cd7d36 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:37.813042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3fea3-b4b8-46c5-a84a-152f96cd7d36 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:37.813042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4443-19e6-4ad0-80ad-9e91b3b3bf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:37.813042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4443-19e6-4ad0-80ad-9e91b3b3bf50 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:37.813042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4443-19e6-4ad0-80ad-9e91b3b3bf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:37.824923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a4443-19e6-4ad0-80ad-9e91b3b3bf50 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:37.824923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4443-19e6-4ad0-80ad-9e91b3b3bf50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:37.824923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a4443-19e6-4ad0-80ad-9e91b3b3bf50 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:37.824923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 814/1634] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:37 2022] POST /identity/v3/projects => generated 405 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:58:37.830034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184a122-5b5f-4200-b45b-c2dcdd5d1f95 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:37.853282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184a122-5b5f-4200-b45b-c2dcdd5d1f95 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:37.989779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b68acb-085b-4c68-8af7-030801ef15b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:37.990502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68acb-085b-4c68-8af7-030801ef15b0 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:37.990779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68acb-085b-4c68-8af7-030801ef15b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:37.991058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b68acb-085b-4c68-8af7-030801ef15b0 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:37.991837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b68acb-085b-4c68-8af7-030801ef15b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:38.001181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b68acb-085b-4c68-8af7-030801ef15b0 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:38.150747 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 821/1635] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:37 2022] POST /identity/v3/users => generated 309 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:38.156946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4443-19e6-4ad0-80ad-9e91b3b3bf50 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:38.172176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a4443-19e6-4ad0-80ad-9e91b3b3bf50 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:38.182162 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a119e75-38a2-4bd5-b746-ee145c27bb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:38.183755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a119e75-38a2-4bd5-b746-ee145c27bb14 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:38.184291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a119e75-38a2-4bd5-b746-ee145c27bb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:38.184725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a119e75-38a2-4bd5-b746-ee145c27bb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:38.185744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a119e75-38a2-4bd5-b746-ee145c27bb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:38.188844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a119e75-38a2-4bd5-b746-ee145c27bb14 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:38.206848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 815/1636] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:38.210899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b68acb-085b-4c68-8af7-030801ef15b0 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:38.225652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b68acb-085b-4c68-8af7-030801ef15b0 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:38.232905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2031375-eb6c-48d1-a839-7ba0162824f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:38.234172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2031375-eb6c-48d1-a839-7ba0162824f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:38.234789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2031375-eb6c-48d1-a839-7ba0162824f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:38.235329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2031375-eb6c-48d1-a839-7ba0162824f7 admin admin] PATH_INFO: `/v3/projects/bf7b5e7d8fb748b6a8c11bcf4b1f948f/users/d3c57b6b2a9c41bd893bb01642556e52/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:38.238881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:38.263211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2031375-eb6c-48d1-a839-7ba0162824f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf7b5e7d8fb748b6a8c11bcf4b1f948f, user_id=d3c57b6b2a9c41bd893bb01642556e52, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:38.266209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2031375-eb6c-48d1-a839-7ba0162824f7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:38.393702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 822/1637] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:38 2022] PUT /identity/v3/projects/bf7b5e7d8fb748b6a8c11bcf4b1f948f/users/d3c57b6b2a9c41bd893bb01642556e52/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:38.400988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a119e75-38a2-4bd5-b746-ee145c27bb14 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:38.407045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:38.416090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a119e75-38a2-4bd5-b746-ee145c27bb14 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:38.544420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b934b-1093-4304-b885-354bc2e97a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:38.545104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b934b-1093-4304-b885-354bc2e97a96 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:38.545351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b934b-1093-4304-b885-354bc2e97a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:38.545630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b934b-1093-4304-b885-354bc2e97a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:38.549468 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b934b-1093-4304-b885-354bc2e97a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:38.551936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b934b-1093-4304-b885-354bc2e97a96 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:38.577091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 816/1638] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:38.580739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2031375-eb6c-48d1-a839-7ba0162824f7 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:38.596949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2031375-eb6c-48d1-a839-7ba0162824f7 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:38.608659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ce592-133f-4ac6-b1e8-993360427bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:38.610636 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ce592-133f-4ac6-b1e8-993360427bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:38.610930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ce592-133f-4ac6-b1e8-993360427bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:38.611212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ce592-133f-4ac6-b1e8-993360427bfc admin admin] PATH_INFO: `/v3/projects/bf7b5e7d8fb748b6a8c11bcf4b1f948f/users/d3c57b6b2a9c41bd893bb01642556e52/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:38.616693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ce592-133f-4ac6-b1e8-993360427bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf7b5e7d8fb748b6a8c11bcf4b1f948f, user_id=d3c57b6b2a9c41bd893bb01642556e52, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:38.620092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ce592-133f-4ac6-b1e8-993360427bfc admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:38.651101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 823/1639] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:38 2022] PUT /identity/v3/projects/bf7b5e7d8fb748b6a8c11bcf4b1f948f/users/d3c57b6b2a9c41bd893bb01642556e52/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:38.908576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed42325-4ed7-4165-a1d6-d30ca3fc9361 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:38.909900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed42325-4ed7-4165-a1d6-d30ca3fc9361 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:38.909900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed42325-4ed7-4165-a1d6-d30ca3fc9361 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:38.911953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 817/1640] 10.222.0.17 () {54 vars in 962 bytes} [Mon Jul 25 05:58:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:58:38.921047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac80d-860b-4838-bada-b0c09d95dd63 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:38.921411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac80d-860b-4838-bada-b0c09d95dd63 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:38.921955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac80d-860b-4838-bada-b0c09d95dd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:38.967987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1d9ac80d-860b-4838-bada-b0c09d95dd63 None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:38.970229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1d9ac80d-860b-4838-bada-b0c09d95dd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:39.121120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 824/1641] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Jul 25 05:58:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:39.130795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774e8cc-1893-4d5a-82a0-dbf7daaae425 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:39.131090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774e8cc-1893-4d5a-82a0-dbf7daaae425 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:39.132368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774e8cc-1893-4d5a-82a0-dbf7daaae425 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:39.137198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 818/1642] 10.222.0.17 () {54 vars in 912 bytes} [Mon Jul 25 05:58:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:58:39.142622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99568400-8eca-4dbc-9a9b-1f6ea4f1d48b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:39.142622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99568400-8eca-4dbc-9a9b-1f6ea4f1d48b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:39.142843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99568400-8eca-4dbc-9a9b-1f6ea4f1d48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:39.177491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-99568400-8eca-4dbc-9a9b-1f6ea4f1d48b None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:39.180098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-99568400-8eca-4dbc-9a9b-1f6ea4f1d48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:39.208303 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 819/1643] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 05:58:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:58:39.215688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ac80d-860b-4838-bada-b0c09d95dd63 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:39.316131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ac80d-860b-4838-bada-b0c09d95dd63 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:39.321422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136b36d-8bff-4af4-ae71-d63abdfe4d58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:39.322111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b36d-8bff-4af4-ae71-d63abdfe4d58 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:39.322386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b36d-8bff-4af4-ae71-d63abdfe4d58 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:39.322696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b36d-8bff-4af4-ae71-d63abdfe4d58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:39.334208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136b36d-8bff-4af4-ae71-d63abdfe4d58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:39.335714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136b36d-8bff-4af4-ae71-d63abdfe4d58 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:39.350842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 825/1644] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:43.347924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26d01-c29d-4f3a-87e9-63f266f73e5b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:43.348414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26d01-c29d-4f3a-87e9-63f266f73e5b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:43.348414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26d01-c29d-4f3a-87e9-63f266f73e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:43.393221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-86e26d01-c29d-4f3a-87e9-63f266f73e5b None None] MFA Rules not processed for user `d3c57b6b2a9c41bd893bb01642556e52`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:43.394492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-86e26d01-c29d-4f3a-87e9-63f266f73e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:43.601819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 820/1645] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:47.041577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0eba8-f659-4071-8bf0-f35a60b32612 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:47.042678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0eba8-f659-4071-8bf0-f35a60b32612 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:47.050189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0eba8-f659-4071-8bf0-f35a60b32612 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:47.050189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 826/1646] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 05:58:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:58:47.068748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3735a-df4b-4d03-a962-898a40958dbc None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:47.069234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3735a-df4b-4d03-a962-898a40958dbc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:47.069575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3735a-df4b-4d03-a962-898a40958dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:47.098198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-b9e3735a-df4b-4d03-a962-898a40958dbc None None] MFA Rules not processed for user `dbfcf2e915c049f08a9e41c555401a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:47.100143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b9e3735a-df4b-4d03-a962-898a40958dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:47.244532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 821/1647] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 05:58:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:47.289150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c5210-7e6c-4f7a-8f20-5ca3b379b193 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:47.289527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c5210-7e6c-4f7a-8f20-5ca3b379b193 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:47.289976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c5210-7e6c-4f7a-8f20-5ca3b379b193 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:47.292833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 827/1648] 10.222.0.17 () {54 vars in 961 bytes} [Mon Jul 25 05:58:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:58:47.306805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c359ae6-29b7-425b-a4db-17f136927355 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:47.307333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c359ae6-29b7-425b-a4db-17f136927355 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:47.307768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c359ae6-29b7-425b-a4db-17f136927355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:47.369599 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-4c359ae6-29b7-425b-a4db-17f136927355 None None] MFA Rules not processed for user `dbfcf2e915c049f08a9e41c555401a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:47.374196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4c359ae6-29b7-425b-a4db-17f136927355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:47.399956 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 822/1649] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Jul 25 05:58:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:47.407610 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c5210-7e6c-4f7a-8f20-5ca3b379b193 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:47.433199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c5210-7e6c-4f7a-8f20-5ca3b379b193 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:47.447181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44492391-6406-4aa6-a791-e5ec01834e83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:47.448044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44492391-6406-4aa6-a791-e5ec01834e83 service placement] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:47.449233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44492391-6406-4aa6-a791-e5ec01834e83 service placement] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:47.450930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44492391-6406-4aa6-a791-e5ec01834e83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:47.473578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44492391-6406-4aa6-a791-e5ec01834e83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:47.476312 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44492391-6406-4aa6-a791-e5ec01834e83 service placement] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:47.497169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 828/1650] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:48.995600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0f5f8-d57c-4a72-bb50-238359247dda None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:48.996176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0f5f8-d57c-4a72-bb50-238359247dda None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:48.996176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0f5f8-d57c-4a72-bb50-238359247dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:49.036550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:49.082166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-fdc0f5f8-d57c-4a72-bb50-238359247dda None None] MFA Rules not processed for user `13e5d648af294c7cbc86ccd362914728`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:49.086393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-fdc0f5f8-d57c-4a72-bb50-238359247dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:49.344732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 823/1651] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:58:49.355172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a42b7-f26c-4425-bf61-f21daf1d6181 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:49.355769 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a42b7-f26c-4425-bf61-f21daf1d6181 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:49.356024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a42b7-f26c-4425-bf61-f21daf1d6181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:49.405130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-bb8a42b7-f26c-4425-bf61-f21daf1d6181 None None] MFA Rules not processed for user `13e5d648af294c7cbc86ccd362914728`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:49.406368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-bb8a42b7-f26c-4425-bf61-f21daf1d6181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:49.441848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 829/1652] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:49.456349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0f5f8-d57c-4a72-bb50-238359247dda None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:49.503911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0f5f8-d57c-4a72-bb50-238359247dda None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:49.515380 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc63abd-2650-414b-bdb5-9d816dae9a00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:49.516239 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc63abd-2650-414b-bdb5-9d816dae9a00 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:49.516488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc63abd-2650-414b-bdb5-9d816dae9a00 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:49.516773 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc63abd-2650-414b-bdb5-9d816dae9a00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:49.546037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc63abd-2650-414b-bdb5-9d816dae9a00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:49.548283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc63abd-2650-414b-bdb5-9d816dae9a00 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:49.591323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 824/1653] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:50.762018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a10e1d-ca60-4036-a0a1-6218bb842a3a None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:50.762506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a10e1d-ca60-4036-a0a1-6218bb842a3a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:50.766310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a10e1d-ca60-4036-a0a1-6218bb842a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:50.781294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ae87-8c1b-467c-87c5-4ba6c8515658 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:50.782323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ae87-8c1b-467c-87c5-4ba6c8515658 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:50.782406 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ae87-8c1b-467c-87c5-4ba6c8515658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:50.819297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-98a10e1d-ca60-4036-a0a1-6218bb842a3a None None] MFA Rules not processed for user `7f269cd38f734209ba0496cf70469608`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:50.823133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-98a10e1d-ca60-4036-a0a1-6218bb842a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:50.841030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8637ae87-8c1b-467c-87c5-4ba6c8515658 None None] MFA Rules not processed for user `6dc49525e3394668b4f1610c4badc144`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:50.842740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8637ae87-8c1b-467c-87c5-4ba6c8515658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:50.900221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:51.001181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 830/1654] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:51.002456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a10e1d-ca60-4036-a0a1-6218bb842a3a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:51.008762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 825/1655] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:58:51.010357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0872c-b3e2-4f1a-94b6-0082e948456b None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:51.011067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0872c-b3e2-4f1a-94b6-0082e948456b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:51.011497 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0872c-b3e2-4f1a-94b6-0082e948456b None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:51.013984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 826/1656] 10.222.0.17 () {54 vars in 912 bytes} [Mon Jul 25 05:58:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:58:51.014821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0872c-b3e2-4f1a-94b6-0082e948456b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:51.055325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0872c-b3e2-4f1a-94b6-0082e948456b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:51.065942 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1597c63-629e-4598-8cc2-3dc23a3b8247 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:51.066658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1597c63-629e-4598-8cc2-3dc23a3b8247 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:51.067110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1597c63-629e-4598-8cc2-3dc23a3b8247 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:51.067552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1597c63-629e-4598-8cc2-3dc23a3b8247 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:51.082208 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1597c63-629e-4598-8cc2-3dc23a3b8247 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:51.084147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1597c63-629e-4598-8cc2-3dc23a3b8247 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:51.099267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a10e1d-ca60-4036-a0a1-6218bb842a3a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:51.105639 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:51.106764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:51.107202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:51.107730 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] PATH_INFO: `/v3/users/2f35c7969e534c61aa45efff90cc56bb` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:51.109560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 827/1657] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:51.111296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f35c7969e534c61aa45efff90cc56bb)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:51.112768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be37cae-20b9-400a-ab6c-585cdc104841 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:51.115694 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be37cae-20b9-400a-ab6c-585cdc104841 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:51.120694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:51.120825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be37cae-20b9-400a-ab6c-585cdc104841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:51.204006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-9be37cae-20b9-400a-ab6c-585cdc104841 None None] MFA Rules not processed for user `ecd61347f8874b89aada19772ec56179`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:51.205761 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9be37cae-20b9-400a-ab6c-585cdc104841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:51.315172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:51.330106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 828/1658] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 05:58:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:51.347975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be37cae-20b9-400a-ab6c-585cdc104841 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:51.360318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be37cae-20b9-400a-ab6c-585cdc104841 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:51.368985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f35c7969e534c61aa45efff90cc56bb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:51.370058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9467aa8-f5ee-4450-8750-e426a063833d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:51.371284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9467aa8-f5ee-4450-8750-e426a063833d service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:51.372969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9467aa8-f5ee-4450-8750-e426a063833d service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:51.373423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9467aa8-f5ee-4450-8750-e426a063833d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:51.377806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f35c7969e534c61aa45efff90cc56bb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:51.388074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9467aa8-f5ee-4450-8750-e426a063833d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:51.389594 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9467aa8-f5ee-4450-8750-e426a063833d service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:51.414223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 829/1659] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:58:51.415150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9467aa8-f5ee-4450-8750-e426a063833d service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:51.443813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9467aa8-f5ee-4450-8750-e426a063833d service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:51.449886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ce547-b579-4753-9d75-b7eff9029a02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:51.450752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ce547-b579-4753-9d75-b7eff9029a02 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:51.451040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ce547-b579-4753-9d75-b7eff9029a02 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:51.451320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ce547-b579-4753-9d75-b7eff9029a02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:51.461408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ce547-b579-4753-9d75-b7eff9029a02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:51.463243 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ce547-b579-4753-9d75-b7eff9029a02 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:51.470936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f35c7969e534c61aa45efff90cc56bb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:51.471578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f35c7969e534c61aa45efff90cc56bb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:51.472682 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f35c7969e534c61aa45efff90cc56bb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:51.506689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 830/1660] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:51.508315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab9400-f00f-41f5-9500-49248612bdb4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:51.510029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab9400-f00f-41f5-9500-49248612bdb4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:51.510649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab9400-f00f-41f5-9500-49248612bdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:51.532457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 831/1661] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:58:51 2022] DELETE /identity/v3/users/2f35c7969e534c61aa45efff90cc56bb => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:51.566008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-87ab9400-f00f-41f5-9500-49248612bdb4 None None] MFA Rules not processed for user `dc8b3ecc32c14e31b55156586129a40c`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:51.567354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-87ab9400-f00f-41f5-9500-49248612bdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:51.743006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 831/1662] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:51.897179 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:51.900011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:52.053685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:52.082145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b126468-c148-42c5-aca4-b13d99eeee72 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:52.090870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47161112-f089-41f6-b543-b67390ea91e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:52.092064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47161112-f089-41f6-b543-b67390ea91e5 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:52.092493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47161112-f089-41f6-b543-b67390ea91e5 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:52.094532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47161112-f089-41f6-b543-b67390ea91e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:52.112116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47161112-f089-41f6-b543-b67390ea91e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:52.114115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47161112-f089-41f6-b543-b67390ea91e5 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:52.126964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 832/1663] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:52.148387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab9400-f00f-41f5-9500-49248612bdb4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:52.149995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-87ab9400-f00f-41f5-9500-49248612bdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:52.297151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab9400-f00f-41f5-9500-49248612bdb4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:52.302766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:52.303944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:52.304346 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:52.304730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] PATH_INFO: `/v3/projects/73471a51fe474dc18527058f0d0c899d` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:52.306348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=73471a51fe474dc18527058f0d0c899d)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:52.308453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:52.527623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73471a51fe474dc18527058f0d0c899d', 'request_id': 'req-0a93ba4a-cb76-4af2-8989-137c7efa49eb'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:52.568343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47161112-f089-41f6-b543-b67390ea91e5 service nova] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:52.569873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73471a51fe474dc18527058f0d0c899d', 'request_id': 'req-0a93ba4a-cb76-4af2-8989-137c7efa49eb'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:52.570006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-47161112-f089-41f6-b543-b67390ea91e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:52.584448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:52.589859 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73471a51fe474dc18527058f0d0c899d', 'request_id': 'req-0a93ba4a-cb76-4af2-8989-137c7efa49eb'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:52.619762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47161112-f089-41f6-b543-b67390ea91e5 service nova] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:52.694794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] The token cache is being invalidate because project 73471a51fe474dc18527058f0d0c899d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:58:52.695218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:52.697363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:52.700820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 832/1664] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:58:52 2022] DELETE /identity/v3/projects/73471a51fe474dc18527058f0d0c899d => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:52.710733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:52.713233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:52.893699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:52.894923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:52.897771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:52.898833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] PATH_INFO: `/v3/users/dcec6f4073664b04a33e6e04fb29eb37` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:52.903064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcec6f4073664b04a33e6e04fb29eb37)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:52.915457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:52.941579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93ba4a-cb76-4af2-8989-137c7efa49eb admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:52.951744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:52.952906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:52.953190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:52.953488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] PATH_INFO: `/v3/users/d3c57b6b2a9c41bd893bb01642556e52` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:52.955164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c57b6b2a9c41bd893bb01642556e52)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:52.956684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: WARNING py.warnings [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:58:52.956684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: warnings.warn(msg) Jul 25 05:58:52.956684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: Jul 25 05:58:52.957445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:53.197273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcec6f4073664b04a33e6e04fb29eb37'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:53.220668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcec6f4073664b04a33e6e04fb29eb37'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:53.312800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c57b6b2a9c41bd893bb01642556e52'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:53.322419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcec6f4073664b04a33e6e04fb29eb37'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:53.323802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcec6f4073664b04a33e6e04fb29eb37'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:53.324641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5cbf5a61-497e-4479-a239-bd8e80e4d17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcec6f4073664b04a33e6e04fb29eb37'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:53.341990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c57b6b2a9c41bd893bb01642556e52'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:53.343615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c57b6b2a9c41bd893bb01642556e52'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:53.372511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 833/1665] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:58:52 2022] DELETE /identity/v3/users/dcec6f4073664b04a33e6e04fb29eb37 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:53.374243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dce651-28e8-4fc9-a98a-50ae1073ed55 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:53.374874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dce651-28e8-4fc9-a98a-50ae1073ed55 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:53.375513 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dce651-28e8-4fc9-a98a-50ae1073ed55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:53.482652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-28dce651-28e8-4fc9-a98a-50ae1073ed55 None None] MFA Rules not processed for user `3bc10b641e34490a83bc587262f4c61a`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:53.483892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-28dce651-28e8-4fc9-a98a-50ae1073ed55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:53.484153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c57b6b2a9c41bd893bb01642556e52'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:53.486206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f5ab315b-144d-44c6-b43d-2a84ed5e3d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c57b6b2a9c41bd893bb01642556e52'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:53.633157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 833/1666] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:58:52 2022] DELETE /identity/v3/users/d3c57b6b2a9c41bd893bb01642556e52 => generated 0 bytes in 923 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:53.634774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0640e5-0ffd-4299-9cdb-e3e98625583c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:53.641884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0640e5-0ffd-4299-9cdb-e3e98625583c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:53.642234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0640e5-0ffd-4299-9cdb-e3e98625583c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:53.671645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:53.734108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3d0640e5-0ffd-4299-9cdb-e3e98625583c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:53.741190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3d0640e5-0ffd-4299-9cdb-e3e98625583c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:53.763102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 834/1667] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:53.776756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1d91-c1fe-41c7-b89c-63cbcb615623 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:53.777245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1d91-c1fe-41c7-b89c-63cbcb615623 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:53.777528 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1d91-c1fe-41c7-b89c-63cbcb615623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:53.869971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-85bf1d91-c1fe-41c7-b89c-63cbcb615623 None None] MFA Rules not processed for user `3bc10b641e34490a83bc587262f4c61a`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:53.874956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-85bf1d91-c1fe-41c7-b89c-63cbcb615623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:53.976044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 834/1668] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:53.976714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0640e5-0ffd-4299-9cdb-e3e98625583c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:53.978061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3d0640e5-0ffd-4299-9cdb-e3e98625583c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:54.013664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:54.068093 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 835/1669] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:58:54.079732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af34b4-7cf6-42d7-8d61-6c5129ac3eb7 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:54.079732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af34b4-7cf6-42d7-8d61-6c5129ac3eb7 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:54.079732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af34b4-7cf6-42d7-8d61-6c5129ac3eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:54.134683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-83af34b4-7cf6-42d7-8d61-6c5129ac3eb7 None None] MFA Rules not processed for user `13e5d648af294c7cbc86ccd362914728`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:54.137344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-83af34b4-7cf6-42d7-8d61-6c5129ac3eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:54.190769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0640e5-0ffd-4299-9cdb-e3e98625583c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:54.214742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0796c-725d-47bb-ab07-3a737f93fd31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:54.215773 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0796c-725d-47bb-ab07-3a737f93fd31 service placement] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:54.217005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0796c-725d-47bb-ab07-3a737f93fd31 service placement] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:54.218616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0796c-725d-47bb-ab07-3a737f93fd31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:54.224485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3ca0796c-725d-47bb-ab07-3a737f93fd31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:54.262419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0796c-725d-47bb-ab07-3a737f93fd31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:54.264743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0796c-725d-47bb-ab07-3a737f93fd31 service placement] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:54.291123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 835/1670] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:54.292383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0796c-725d-47bb-ab07-3a737f93fd31 service placement] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:54.296456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3ca0796c-725d-47bb-ab07-3a737f93fd31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:54.298540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 836/1671] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:54.299540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af34b4-7cf6-42d7-8d61-6c5129ac3eb7 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:54.303062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-83af34b4-7cf6-42d7-8d61-6c5129ac3eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:54.436711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af34b4-7cf6-42d7-8d61-6c5129ac3eb7 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:54.448211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0796c-725d-47bb-ab07-3a737f93fd31 service placement] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:54.449321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80309dc0-98ea-47bf-91a4-bfa1100881d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:54.450334 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80309dc0-98ea-47bf-91a4-bfa1100881d6 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:54.451504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80309dc0-98ea-47bf-91a4-bfa1100881d6 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:54.452166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80309dc0-98ea-47bf-91a4-bfa1100881d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:54.457210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cc9e1-cc14-44ca-9ebd-7b431b75ab37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:54.458944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cc9e1-cc14-44ca-9ebd-7b431b75ab37 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:54.460547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cc9e1-cc14-44ca-9ebd-7b431b75ab37 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:54.460912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cc9e1-cc14-44ca-9ebd-7b431b75ab37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:54.475201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80309dc0-98ea-47bf-91a4-bfa1100881d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:54.479636 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80309dc0-98ea-47bf-91a4-bfa1100881d6 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:54.483446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cc9e1-cc14-44ca-9ebd-7b431b75ab37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:54.488238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cc9e1-cc14-44ca-9ebd-7b431b75ab37 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:54.501574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 837/1672] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:54.507176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 836/1673] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:54.734007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80309dc0-98ea-47bf-91a4-bfa1100881d6 service nova] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:54.736931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-80309dc0-98ea-47bf-91a4-bfa1100881d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:54.759590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80309dc0-98ea-47bf-91a4-bfa1100881d6 service nova] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:54.776010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:54.776958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:54.777245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:54.777520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] PATH_INFO: `/v3/projects/bf7b5e7d8fb748b6a8c11bcf4b1f948f` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:54.783675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf7b5e7d8fb748b6a8c11bcf4b1f948f)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:54.783798 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: WARNING py.warnings [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:58:54.783798 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: warnings.warn(msg) Jul 25 05:58:54.783798 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: Jul 25 05:58:54.784974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:54.838070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:54.980398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf7b5e7d8fb748b6a8c11bcf4b1f948f', 'request_id': 'req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:55.009901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf7b5e7d8fb748b6a8c11bcf4b1f948f', 'request_id': 'req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:55.026255 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf7b5e7d8fb748b6a8c11bcf4b1f948f', 'request_id': 'req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:55.117369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] The token cache is being invalidate because project bf7b5e7d8fb748b6a8c11bcf4b1f948f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:58:55.118289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:55.127133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f9a75ce1-69ad-42ab-8c1a-5a535a29c0dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:55.130335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 838/1674] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:58:54 2022] DELETE /identity/v3/projects/bf7b5e7d8fb748b6a8c11bcf4b1f948f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:55.135550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29a9f0-6717-4d2c-9436-5efee388a38c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:55.139590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29a9f0-6717-4d2c-9436-5efee388a38c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:55.140076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29a9f0-6717-4d2c-9436-5efee388a38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:55.180823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0e29a9f0-6717-4d2c-9436-5efee388a38c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:55.182336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0e29a9f0-6717-4d2c-9436-5efee388a38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:55.429434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 837/1675] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:55.436972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe46c0-49ac-4760-80a2-dd1e7858393a None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:55.437499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe46c0-49ac-4760-80a2-dd1e7858393a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:55.438720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe46c0-49ac-4760-80a2-dd1e7858393a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:55.477071 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-06fe46c0-49ac-4760-80a2-dd1e7858393a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:55.481106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-06fe46c0-49ac-4760-80a2-dd1e7858393a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:55.500988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 839/1676] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:55.504202 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29a9f0-6717-4d2c-9436-5efee388a38c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:55.515177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29a9f0-6717-4d2c-9436-5efee388a38c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:55.539485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b57a43-76dd-46e1-84c9-667c1079164e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:55.540626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b57a43-76dd-46e1-84c9-667c1079164e admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:55.541076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b57a43-76dd-46e1-84c9-667c1079164e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:55.541562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b57a43-76dd-46e1-84c9-667c1079164e admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:55.542611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b57a43-76dd-46e1-84c9-667c1079164e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:58:55.543111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b57a43-76dd-46e1-84c9-667c1079164e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:55.545208 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b57a43-76dd-46e1-84c9-667c1079164e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:55.566678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 838/1677] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:58:55.579353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe46c0-49ac-4760-80a2-dd1e7858393a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:55.610361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe46c0-49ac-4760-80a2-dd1e7858393a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:55.647170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655f6c0-52c5-434f-8e26-7ed62c711e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:55.648453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655f6c0-52c5-434f-8e26-7ed62c711e6b admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:55.649997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655f6c0-52c5-434f-8e26-7ed62c711e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:55.650362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655f6c0-52c5-434f-8e26-7ed62c711e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:55.662946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655f6c0-52c5-434f-8e26-7ed62c711e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:55.668159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655f6c0-52c5-434f-8e26-7ed62c711e6b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:55.674273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4daef5-f7e3-42e7-a6a9-6039b2cc583c None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:55.674837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4daef5-f7e3-42e7-a6a9-6039b2cc583c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:55.675405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4daef5-f7e3-42e7-a6a9-6039b2cc583c None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:55.677256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 839/1678] 10.222.0.17 () {54 vars in 912 bytes} [Mon Jul 25 05:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:58:55.684319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdec3f4-99d9-43e7-b18d-866707acf715 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:55.701194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdec3f4-99d9-43e7-b18d-866707acf715 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:55.701194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdec3f4-99d9-43e7-b18d-866707acf715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:55.755537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-fcdec3f4-99d9-43e7-b18d-866707acf715 None None] MFA Rules not processed for user `ecd61347f8874b89aada19772ec56179`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:55.761547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-fcdec3f4-99d9-43e7-b18d-866707acf715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:55.792884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:55.869619 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 840/1679] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:55 2022] POST /identity/v3/projects => generated 365 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:55.874440 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655f6c0-52c5-434f-8e26-7ed62c711e6b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:55.982986 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 840/1680] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 05:58:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:56.059350 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655f6c0-52c5-434f-8e26-7ed62c711e6b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:56.064793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30fc51-371f-43a6-a348-1c708852a048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:56.065851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30fc51-371f-43a6-a348-1c708852a048 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:56.066339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30fc51-371f-43a6-a348-1c708852a048 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:56.066864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30fc51-371f-43a6-a348-1c708852a048 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:56.067809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30fc51-371f-43a6-a348-1c708852a048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:56.070044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30fc51-371f-43a6-a348-1c708852a048 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:56.155277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdec3f4-99d9-43e7-b18d-866707acf715 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:56.157293 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-fcdec3f4-99d9-43e7-b18d-866707acf715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:56.172619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdec3f4-99d9-43e7-b18d-866707acf715 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:56.237810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 841/1681] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:55 2022] POST /identity/v3/users => generated 283 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:58:56.242884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30fc51-371f-43a6-a348-1c708852a048 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:56.264777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30fc51-371f-43a6-a348-1c708852a048 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:56.281761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cde873-2bd8-4817-a05f-19c6282affa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:56.283781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde873-2bd8-4817-a05f-19c6282affa5 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:56.284066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde873-2bd8-4817-a05f-19c6282affa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:56.284593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde873-2bd8-4817-a05f-19c6282affa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:56.286159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cde873-2bd8-4817-a05f-19c6282affa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:56.288379 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cde873-2bd8-4817-a05f-19c6282affa5 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:56.316512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 842/1682] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:56.323086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cde873-2bd8-4817-a05f-19c6282affa5 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:56.338991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cde873-2bd8-4817-a05f-19c6282affa5 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:56.355600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3383fa-45df-4b91-87c6-ab93e8a78e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:56.356645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3383fa-45df-4b91-87c6-ab93e8a78e6a admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:56.357153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3383fa-45df-4b91-87c6-ab93e8a78e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:56.357734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3383fa-45df-4b91-87c6-ab93e8a78e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:56.358748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3383fa-45df-4b91-87c6-ab93e8a78e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:56.361886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3383fa-45df-4b91-87c6-ab93e8a78e6a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:56.388811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 843/1683] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:56.394562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3383fa-45df-4b91-87c6-ab93e8a78e6a admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:56.404595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee2501-4966-4264-8c79-500fe41943f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:56.405552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee2501-4966-4264-8c79-500fe41943f4 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:56.405978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee2501-4966-4264-8c79-500fe41943f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:56.407007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee2501-4966-4264-8c79-500fe41943f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:56.417445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3383fa-45df-4b91-87c6-ab93e8a78e6a admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:56.424805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee2501-4966-4264-8c79-500fe41943f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:56.426501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee2501-4966-4264-8c79-500fe41943f4 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:56.429575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a6054-0e7c-4ae1-a750-59c18b562bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:56.432785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a6054-0e7c-4ae1-a750-59c18b562bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:56.434563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a6054-0e7c-4ae1-a750-59c18b562bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:56.434964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a6054-0e7c-4ae1-a750-59c18b562bd9 admin admin] PATH_INFO: `/v3/projects/490373e2a9ff4e14ba35a2326aa6b9ba/users/de1e921893784c26930d63651b74be7c/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:56.439105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 841/1684] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:56.469694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a6054-0e7c-4ae1-a750-59c18b562bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=490373e2a9ff4e14ba35a2326aa6b9ba, user_id=de1e921893784c26930d63651b74be7c, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:56.473078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a6054-0e7c-4ae1-a750-59c18b562bd9 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:56.539382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 844/1685] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:56 2022] PUT /identity/v3/projects/490373e2a9ff4e14ba35a2326aa6b9ba/users/de1e921893784c26930d63651b74be7c/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:56.553730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f608b-9552-4f60-b2a4-96da1a894655 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:56.554234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f608b-9552-4f60-b2a4-96da1a894655 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:56.555818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f608b-9552-4f60-b2a4-96da1a894655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:56.678633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-eb6f608b-9552-4f60-b2a4-96da1a894655 None None] MFA Rules not processed for user `de1e921893784c26930d63651b74be7c`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:56.680287 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-eb6f608b-9552-4f60-b2a4-96da1a894655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:56.941251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 842/1686] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:56.944400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a6054-0e7c-4ae1-a750-59c18b562bd9 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:57.007705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:57.149494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a6054-0e7c-4ae1-a750-59c18b562bd9 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:57.162396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb457de-c43b-452d-8e04-3b3ba7284046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:57.163130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb457de-c43b-452d-8e04-3b3ba7284046 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:57.163420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb457de-c43b-452d-8e04-3b3ba7284046 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:57.163701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb457de-c43b-452d-8e04-3b3ba7284046 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:57.164420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb457de-c43b-452d-8e04-3b3ba7284046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:57.177459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb457de-c43b-452d-8e04-3b3ba7284046 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:57.340128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 845/1687] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:56 2022] POST /identity/v3/projects => generated 365 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:57.341377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f608b-9552-4f60-b2a4-96da1a894655 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:57.366308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb457de-c43b-452d-8e04-3b3ba7284046 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:57.370585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7fb457de-c43b-452d-8e04-3b3ba7284046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:57.380503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f608b-9552-4f60-b2a4-96da1a894655 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:57.565410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb457de-c43b-452d-8e04-3b3ba7284046 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:57.567556 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0ed32-477c-44fa-b845-04025a22f86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:57.569273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ed32-477c-44fa-b845-04025a22f86e admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:57.570800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ed32-477c-44fa-b845-04025a22f86e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:57.571590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:57.572168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ed32-477c-44fa-b845-04025a22f86e admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:57.572661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:57.578406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:57.578558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0ed32-477c-44fa-b845-04025a22f86e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:57.578651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] PATH_INFO: `/v3/projects/801185ee50e945d1a4c1f7bf75694bf5` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:57.578651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] RBAC: Authorizing `identity:delete_project(project_id=801185ee50e945d1a4c1f7bf75694bf5)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:57.578820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0ed32-477c-44fa-b845-04025a22f86e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:57.578918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:57.740537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:57.746977 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 843/1688] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:57 2022] POST /identity/v3/users => generated 289 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:57.750628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0ed32-477c-44fa-b845-04025a22f86e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:57.790454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0ed32-477c-44fa-b845-04025a22f86e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:57.886230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801185ee50e945d1a4c1f7bf75694bf5', 'request_id': 'req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:57.919256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801185ee50e945d1a4c1f7bf75694bf5', 'request_id': 'req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:57.960824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801185ee50e945d1a4c1f7bf75694bf5', 'request_id': 'req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:57.982387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06779f-23e8-47bc-abbe-820da1e71005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:57.983441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06779f-23e8-47bc-abbe-820da1e71005 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:57.983840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06779f-23e8-47bc-abbe-820da1e71005 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:57.984223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06779f-23e8-47bc-abbe-820da1e71005 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:57.985178 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06779f-23e8-47bc-abbe-820da1e71005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:57.987055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06779f-23e8-47bc-abbe-820da1e71005 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:58.020528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 844/1689] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:58.037915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06779f-23e8-47bc-abbe-820da1e71005 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:58.067454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06779f-23e8-47bc-abbe-820da1e71005 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:58.080453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f4796-7aa9-40e6-9f87-b3e8cbea0360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:58.081522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f4796-7aa9-40e6-9f87-b3e8cbea0360 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:58.082172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f4796-7aa9-40e6-9f87-b3e8cbea0360 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:58.084105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f4796-7aa9-40e6-9f87-b3e8cbea0360 admin admin] PATH_INFO: `/v3/projects/b30c7d625bdf499b8aae3edbff9926fa/users/a67e028d57d5486eb0dd922383a10224/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:58.119837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f4796-7aa9-40e6-9f87-b3e8cbea0360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30c7d625bdf499b8aae3edbff9926fa, user_id=a67e028d57d5486eb0dd922383a10224, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:58.125598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f4796-7aa9-40e6-9f87-b3e8cbea0360 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:58.127284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] The token cache is being invalidate because project 801185ee50e945d1a4c1f7bf75694bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:58:58.127734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:58.128562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c6d6d2f1-9bbc-4930-b7d5-6f2e26cc042d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:58:58.130628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 846/1690] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:58:57 2022] DELETE /identity/v3/projects/801185ee50e945d1a4c1f7bf75694bf5 => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:58.137804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bef0f-339c-4ec9-857f-b0dacc50ec24 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:58.148150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bef0f-339c-4ec9-857f-b0dacc50ec24 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:58.148520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bef0f-339c-4ec9-857f-b0dacc50ec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:58.199843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-dc2bef0f-339c-4ec9-857f-b0dacc50ec24 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:58.204519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-dc2bef0f-339c-4ec9-857f-b0dacc50ec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:58.208227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 845/1691] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:58 2022] PUT /identity/v3/projects/b30c7d625bdf499b8aae3edbff9926fa/users/a67e028d57d5486eb0dd922383a10224/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:58.209806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f4796-7aa9-40e6-9f87-b3e8cbea0360 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:58.219561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0f6f4796-7aa9-40e6-9f87-b3e8cbea0360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:58.342825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f4796-7aa9-40e6-9f87-b3e8cbea0360 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:58.350666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6f1b6-6505-4215-bb84-8387709278e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:58.351976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6f1b6-6505-4215-bb84-8387709278e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:58.351976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6f1b6-6505-4215-bb84-8387709278e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:58.352164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6f1b6-6505-4215-bb84-8387709278e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:58.354015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6f1b6-6505-4215-bb84-8387709278e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:58.354500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6f1b6-6505-4215-bb84-8387709278e9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:58.367428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 846/1692] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:58.371147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6f1b6-6505-4215-bb84-8387709278e9 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:58.389348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6f1b6-6505-4215-bb84-8387709278e9 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:58.406061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 847/1693] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:58.410291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da182f-0d91-4e5c-a50d-2478e33ea789 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:58.410847 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da182f-0d91-4e5c-a50d-2478e33ea789 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:58.411151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da182f-0d91-4e5c-a50d-2478e33ea789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:58.436132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5cd55-3078-4b66-8c9c-3af927709a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:58.437096 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5cd55-3078-4b66-8c9c-3af927709a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:58.437283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5cd55-3078-4b66-8c9c-3af927709a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:58.438100 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5cd55-3078-4b66-8c9c-3af927709a1f admin admin] PATH_INFO: `/v3/projects/b30c7d625bdf499b8aae3edbff9926fa/users/a67e028d57d5486eb0dd922383a10224/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:58.443689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5cd55-3078-4b66-8c9c-3af927709a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30c7d625bdf499b8aae3edbff9926fa, user_id=a67e028d57d5486eb0dd922383a10224, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:58.446387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5cd55-3078-4b66-8c9c-3af927709a1f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:58.460559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-61da182f-0d91-4e5c-a50d-2478e33ea789 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:58.462591 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-61da182f-0d91-4e5c-a50d-2478e33ea789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:58.475227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 847/1694] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:58 2022] PUT /identity/v3/projects/b30c7d625bdf499b8aae3edbff9926fa/users/a67e028d57d5486eb0dd922383a10224/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:58.476832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a425e90-a9c7-4b3d-929d-52d694b564ed None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:58.477123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a425e90-a9c7-4b3d-929d-52d694b564ed None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:58.478565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a425e90-a9c7-4b3d-929d-52d694b564ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:58.514322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2a425e90-a9c7-4b3d-929d-52d694b564ed None None] MFA Rules not processed for user `a67e028d57d5486eb0dd922383a10224`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:58.517073 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2a425e90-a9c7-4b3d-929d-52d694b564ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:58.588463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 848/1695] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:58:58.592526 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da182f-0d91-4e5c-a50d-2478e33ea789 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:58.612874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da182f-0d91-4e5c-a50d-2478e33ea789 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:58.619499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df04419-cf76-4713-a370-f8a0d39c4d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:58.620501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df04419-cf76-4713-a370-f8a0d39c4d69 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:58.621043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df04419-cf76-4713-a370-f8a0d39c4d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:58.621468 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df04419-cf76-4713-a370-f8a0d39c4d69 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:58.623225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df04419-cf76-4713-a370-f8a0d39c4d69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:58:58.623612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df04419-cf76-4713-a370-f8a0d39c4d69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:58.626060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df04419-cf76-4713-a370-f8a0d39c4d69 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:58.652532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 849/1696] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:58:58.656465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df04419-cf76-4713-a370-f8a0d39c4d69 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:58.677105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df04419-cf76-4713-a370-f8a0d39c4d69 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:58.683460 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bd331-10d1-4cec-8074-d2e8a3f31b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:58.683994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bd331-10d1-4cec-8074-d2e8a3f31b14 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:58.684353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bd331-10d1-4cec-8074-d2e8a3f31b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:58.684740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bd331-10d1-4cec-8074-d2e8a3f31b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:58.686299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bd331-10d1-4cec-8074-d2e8a3f31b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:58.689373 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bd331-10d1-4cec-8074-d2e8a3f31b14 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:58.695135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:58.812417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 848/1697] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:58.814098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a425e90-a9c7-4b3d-929d-52d694b564ed None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:58.819385 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2a425e90-a9c7-4b3d-929d-52d694b564ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:58.847369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 850/1698] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:58:58 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:58:58.856820 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bd331-10d1-4cec-8074-d2e8a3f31b14 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:58.959937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a425e90-a9c7-4b3d-929d-52d694b564ed None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:59.010117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:59.037331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bd331-10d1-4cec-8074-d2e8a3f31b14 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:59.052799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50c4d7-8606-45e0-9b56-e44df86f151b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:59.054988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50c4d7-8606-45e0-9b56-e44df86f151b admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:59.071476 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50c4d7-8606-45e0-9b56-e44df86f151b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:59.071956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50c4d7-8606-45e0-9b56-e44df86f151b admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:59.072868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50c4d7-8606-45e0-9b56-e44df86f151b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:59.078838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50c4d7-8606-45e0-9b56-e44df86f151b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:59.081785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e30b4-d1e8-4ff0-902d-49546b6d214f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:59.082493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e30b4-d1e8-4ff0-902d-49546b6d214f service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:59.082806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e30b4-d1e8-4ff0-902d-49546b6d214f service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:59.083071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e30b4-d1e8-4ff0-902d-49546b6d214f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:59.221469 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 851/1699] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:58:58 2022] POST /identity/v3/users => generated 283 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:58:59.225768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50c4d7-8606-45e0-9b56-e44df86f151b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:59.233319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e30b4-d1e8-4ff0-902d-49546b6d214f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:59.235343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e30b4-d1e8-4ff0-902d-49546b6d214f service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:59.250905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 849/1700] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:59.264026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50c4d7-8606-45e0-9b56-e44df86f151b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:59.269593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992b5b5-9445-404a-8bb8-45e13c86af65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:59.270288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992b5b5-9445-404a-8bb8-45e13c86af65 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:59.270610 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992b5b5-9445-404a-8bb8-45e13c86af65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:59.270891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992b5b5-9445-404a-8bb8-45e13c86af65 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:59.271638 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992b5b5-9445-404a-8bb8-45e13c86af65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:59.273294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992b5b5-9445-404a-8bb8-45e13c86af65 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:59.288814 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 852/1701] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:59.290429 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e30b4-d1e8-4ff0-902d-49546b6d214f service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:59.342962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e30b4-d1e8-4ff0-902d-49546b6d214f service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:59.365187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37608dec-f23b-430d-9453-07a47ad3eeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:59.377205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37608dec-f23b-430d-9453-07a47ad3eeb0 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:59.378861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37608dec-f23b-430d-9453-07a47ad3eeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:59.380560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37608dec-f23b-430d-9453-07a47ad3eeb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:59.381477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37608dec-f23b-430d-9453-07a47ad3eeb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:59.385907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37608dec-f23b-430d-9453-07a47ad3eeb0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:59.433197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 850/1702] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:58:59.438639 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992b5b5-9445-404a-8bb8-45e13c86af65 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:59.469695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fc739-5528-4d69-afaa-133ad69dd051 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:59.469870 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fc739-5528-4d69-afaa-133ad69dd051 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:59.470154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fc739-5528-4d69-afaa-133ad69dd051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:59.473137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992b5b5-9445-404a-8bb8-45e13c86af65 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:59.480501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225765ff-858d-47e7-af80-709b7f42a5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:59.480501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225765ff-858d-47e7-af80-709b7f42a5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:59.480501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225765ff-858d-47e7-af80-709b7f42a5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:59.481082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225765ff-858d-47e7-af80-709b7f42a5ae admin admin] PATH_INFO: `/v3/projects/76b1bb5d3de346669eeaf8767705e4e9/users/64c3a48be83048b2b6dfd91f284bce57/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:59.499730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-4a3fc739-5528-4d69-afaa-133ad69dd051 None None] MFA Rules not processed for user `6dc49525e3394668b4f1610c4badc144`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:59.501986 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4a3fc739-5528-4d69-afaa-133ad69dd051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:59.509461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225765ff-858d-47e7-af80-709b7f42a5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b1bb5d3de346669eeaf8767705e4e9, user_id=64c3a48be83048b2b6dfd91f284bce57, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:59.512459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225765ff-858d-47e7-af80-709b7f42a5ae admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:59.602527 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 851/1703] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:58:59.612903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 853/1704] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:58:59 2022] PUT /identity/v3/projects/76b1bb5d3de346669eeaf8767705e4e9/users/64c3a48be83048b2b6dfd91f284bce57/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:58:59.625634 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c708b-5919-437f-8ddd-e294b0ca7167 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:59.626018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c708b-5919-437f-8ddd-e294b0ca7167 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:59.626335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c708b-5919-437f-8ddd-e294b0ca7167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:59.673418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-140c708b-5919-437f-8ddd-e294b0ca7167 None None] MFA Rules not processed for user `64c3a48be83048b2b6dfd91f284bce57`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:59.675175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-140c708b-5919-437f-8ddd-e294b0ca7167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:59.682207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335fc6-98fa-4b09-819e-8323d49e9804 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:59.686294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335fc6-98fa-4b09-819e-8323d49e9804 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:59.686294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335fc6-98fa-4b09-819e-8323d49e9804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:59.687571 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:59.795945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9b335fc6-98fa-4b09-819e-8323d49e9804 None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:58:59.796466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 852/1705] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:58:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:59.802110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9b335fc6-98fa-4b09-819e-8323d49e9804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:59.816365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c708b-5919-437f-8ddd-e294b0ca7167 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:58:59.818249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-140c708b-5919-437f-8ddd-e294b0ca7167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:58:59.955315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 854/1706] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 05:58:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:58:59.959386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c708b-5919-437f-8ddd-e294b0ca7167 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:58:59.967478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd0bfa-172c-44cd-87f9-ceed3c730676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:58:59.968053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd0bfa-172c-44cd-87f9-ceed3c730676 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:58:59.969210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd0bfa-172c-44cd-87f9-ceed3c730676 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:58:59.969559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd0bfa-172c-44cd-87f9-ceed3c730676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:58:59.985165 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd0bfa-172c-44cd-87f9-ceed3c730676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:58:59.986670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd0bfa-172c-44cd-87f9-ceed3c730676 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:58:59.999289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 853/1707] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:58:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:00.275857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:00.424216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b335fc6-98fa-4b09-819e-8323d49e9804 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:00.426407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9b335fc6-98fa-4b09-819e-8323d49e9804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:00.582604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81f323-72c5-4ec6-b742-de9665b99af6 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:00.582898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81f323-72c5-4ec6-b742-de9665b99af6 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:00.583361 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81f323-72c5-4ec6-b742-de9665b99af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:00.666049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-7f81f323-72c5-4ec6-b742-de9665b99af6 None None] MFA Rules not processed for user `7f269cd38f734209ba0496cf70469608`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:00.694368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7f81f323-72c5-4ec6-b742-de9665b99af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:00.845803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b335fc6-98fa-4b09-819e-8323d49e9804 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:00.852871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d19a7-de2d-4903-9db0-a92875d862e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:00.853755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d19a7-de2d-4903-9db0-a92875d862e6 service swift] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:00.854748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d19a7-de2d-4903-9db0-a92875d862e6 service swift] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:00.855215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d19a7-de2d-4903-9db0-a92875d862e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:00.875516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d19a7-de2d-4903-9db0-a92875d862e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:00.880819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d19a7-de2d-4903-9db0-a92875d862e6 service swift] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:00.901361 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 854/1708] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:00.908473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 855/1709] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 05:59:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:59:02.607187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81f323-72c5-4ec6-b742-de9665b99af6 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:02.654084 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:02.763156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81f323-72c5-4ec6-b742-de9665b99af6 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:02.771931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:02.773462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:02.773912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:02.774343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] PATH_INFO: `/v3/users/de1e921893784c26930d63651b74be7c` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:02.776705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1e921893784c26930d63651b74be7c)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:02.780640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:03.077352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1e921893784c26930d63651b74be7c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:03.107638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1e921893784c26930d63651b74be7c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:03.107638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1e921893784c26930d63651b74be7c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:03.212393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1e921893784c26930d63651b74be7c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:03.216847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1e921893784c26930d63651b74be7c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:03.271550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 855/1710] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:02 2022] DELETE /identity/v3/users/de1e921893784c26930d63651b74be7c => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:03.280102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3b1ea-df78-46fb-8ff9-06a0c822c6f0 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:03.280102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3b1ea-df78-46fb-8ff9-06a0c822c6f0 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:03.280102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3b1ea-df78-46fb-8ff9-06a0c822c6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:03.314164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-5ba3b1ea-df78-46fb-8ff9-06a0c822c6f0 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:03.316402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5ba3b1ea-df78-46fb-8ff9-06a0c822c6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:03.421987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:03.517180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 856/1711] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:03.571104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:03.572821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:03.841518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff216b2-1e11-4816-9f7a-0cd3d8ad730b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:03.863478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e975a-9432-4e19-9956-089c2673613e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:03.866016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e975a-9432-4e19-9956-089c2673613e service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:03.867620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e975a-9432-4e19-9956-089c2673613e service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:03.871992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e975a-9432-4e19-9956-089c2673613e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:03.899191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e975a-9432-4e19-9956-089c2673613e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:03.901036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e975a-9432-4e19-9956-089c2673613e service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:03.930708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 856/1712] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:04.960724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f6f2e-fd34-4cdd-8813-cb833e5ec879 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:04.961320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f6f2e-fd34-4cdd-8813-cb833e5ec879 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:04.962043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f6f2e-fd34-4cdd-8813-cb833e5ec879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:05.023225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-975f6f2e-fd34-4cdd-8813-cb833e5ec879 None None] MFA Rules not processed for user `dc8b3ecc32c14e31b55156586129a40c`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:05.028042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-975f6f2e-fd34-4cdd-8813-cb833e5ec879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:05.268140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 857/1713] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:05.400063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07648736-b0d2-4cdc-92ed-1ff9d689a41d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:05.400063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07648736-b0d2-4cdc-92ed-1ff9d689a41d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:05.400740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07648736-b0d2-4cdc-92ed-1ff9d689a41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:05.454549 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-07648736-b0d2-4cdc-92ed-1ff9d689a41d None None] MFA Rules not processed for user `ecd61347f8874b89aada19772ec56179`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:05.456366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-07648736-b0d2-4cdc-92ed-1ff9d689a41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:05.493194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 857/1714] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Jul 25 05:59:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:05.559287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f6f2e-fd34-4cdd-8813-cb833e5ec879 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:05.563065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-975f6f2e-fd34-4cdd-8813-cb833e5ec879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:05.605005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f6f2e-fd34-4cdd-8813-cb833e5ec879 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:05.616728 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151f142-33e8-46d1-a5e3-3c5f800f7b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:05.618027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151f142-33e8-46d1-a5e3-3c5f800f7b13 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:05.618836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151f142-33e8-46d1-a5e3-3c5f800f7b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:05.619381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151f142-33e8-46d1-a5e3-3c5f800f7b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:05.643625 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151f142-33e8-46d1-a5e3-3c5f800f7b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:05.649216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151f142-33e8-46d1-a5e3-3c5f800f7b13 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:05.676317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 858/1715] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:05.719269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07648736-b0d2-4cdc-92ed-1ff9d689a41d None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:05.731097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07648736-b0d2-4cdc-92ed-1ff9d689a41d None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:05.760490 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92642d6-d471-4f78-8eb0-2fa3f63c9de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:05.761181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92642d6-d471-4f78-8eb0-2fa3f63c9de6 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:05.761464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92642d6-d471-4f78-8eb0-2fa3f63c9de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:05.761745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92642d6-d471-4f78-8eb0-2fa3f63c9de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:05.804962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92642d6-d471-4f78-8eb0-2fa3f63c9de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:05.806561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92642d6-d471-4f78-8eb0-2fa3f63c9de6 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:05.857621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 858/1716] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:06.442275 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151f142-33e8-46d1-a5e3-3c5f800f7b13 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:06.446903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d151f142-33e8-46d1-a5e3-3c5f800f7b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:06.478669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151f142-33e8-46d1-a5e3-3c5f800f7b13 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:06.485600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:06.489659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:06.490843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:06.491326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] PATH_INFO: `/v3/projects/490373e2a9ff4e14ba35a2326aa6b9ba` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:06.493957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=490373e2a9ff4e14ba35a2326aa6b9ba)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:06.499118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:06.739112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490373e2a9ff4e14ba35a2326aa6b9ba', 'request_id': 'req-98b261dc-30c9-4c8a-9180-bb5df956fad1'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:06.783456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490373e2a9ff4e14ba35a2326aa6b9ba', 'request_id': 'req-98b261dc-30c9-4c8a-9180-bb5df956fad1'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:06.801590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490373e2a9ff4e14ba35a2326aa6b9ba', 'request_id': 'req-98b261dc-30c9-4c8a-9180-bb5df956fad1'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:06.926791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] The token cache is being invalidate because project 490373e2a9ff4e14ba35a2326aa6b9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:06.927090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:06.928217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-98b261dc-30c9-4c8a-9180-bb5df956fad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:06.931759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 859/1717] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:06 2022] DELETE /identity/v3/projects/490373e2a9ff4e14ba35a2326aa6b9ba => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:06.939503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92642d6-d471-4f78-8eb0-2fa3f63c9de6 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:06.968532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92642d6-d471-4f78-8eb0-2fa3f63c9de6 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:07.021548 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:07.162865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:07.167606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:07.168104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:07.169857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] PATH_INFO: `/v3/users/a67e028d57d5486eb0dd922383a10224` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:07.175327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a67e028d57d5486eb0dd922383a10224)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:07.180554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:07.537611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a67e028d57d5486eb0dd922383a10224'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:07.566192 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a67e028d57d5486eb0dd922383a10224'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:07.578009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a67e028d57d5486eb0dd922383a10224'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:07.738783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a67e028d57d5486eb0dd922383a10224'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:07.740938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a63ef22d-4a18-4b8e-81e8-5efabf6702cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a67e028d57d5486eb0dd922383a10224'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:07.775889 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 859/1718] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:06 2022] DELETE /identity/v3/users/a67e028d57d5486eb0dd922383a10224 => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:10.552373 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdacb3a-6334-4bed-8e2d-9cdd14707d89 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:10.552914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdacb3a-6334-4bed-8e2d-9cdd14707d89 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:10.552914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdacb3a-6334-4bed-8e2d-9cdd14707d89 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:10.554831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 860/1719] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 05:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:59:10.647152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0df26-8a8f-4468-bcdf-efad0cbda790 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:10.647152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0df26-8a8f-4468-bcdf-efad0cbda790 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:10.647152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0df26-8a8f-4468-bcdf-efad0cbda790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:10.696222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-cec0df26-8a8f-4468-bcdf-efad0cbda790 None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:10.697872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-cec0df26-8a8f-4468-bcdf-efad0cbda790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:10.816363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdacb3a-6334-4bed-8e2d-9cdd14707d89 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:10.818883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3bdacb3a-6334-4bed-8e2d-9cdd14707d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:10.936257 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 860/1720] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 05:59:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:10.979108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:11.011457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdacb3a-6334-4bed-8e2d-9cdd14707d89 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:11.023833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:11.023833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:11.023833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:11.023833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] PATH_INFO: `/v3/projects/b30c7d625bdf499b8aae3edbff9926fa` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:11.024182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30c7d625bdf499b8aae3edbff9926fa)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:11.026135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:11.054520 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0df26-8a8f-4468-bcdf-efad0cbda790 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:11.056681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-cec0df26-8a8f-4468-bcdf-efad0cbda790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:11.078557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0df26-8a8f-4468-bcdf-efad0cbda790 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:11.102370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2294f0-0947-4f2b-b3fc-81a55f33f3fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:11.106350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2294f0-0947-4f2b-b3fc-81a55f33f3fe service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:11.106676 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2294f0-0947-4f2b-b3fc-81a55f33f3fe service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:11.106961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2294f0-0947-4f2b-b3fc-81a55f33f3fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:11.118968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2294f0-0947-4f2b-b3fc-81a55f33f3fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:11.120778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2294f0-0947-4f2b-b3fc-81a55f33f3fe service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:11.162866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 861/1721] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:11.178993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30c7d625bdf499b8aae3edbff9926fa', 'request_id': 'req-66243aa9-81c3-4144-8067-0d7d8f1255fb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:11.189355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30c7d625bdf499b8aae3edbff9926fa', 'request_id': 'req-66243aa9-81c3-4144-8067-0d7d8f1255fb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:11.201440 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30c7d625bdf499b8aae3edbff9926fa', 'request_id': 'req-66243aa9-81c3-4144-8067-0d7d8f1255fb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:11.284981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a547a-726a-40a1-8718-2b31521b4295 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:11.285275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a547a-726a-40a1-8718-2b31521b4295 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:11.285561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a547a-726a-40a1-8718-2b31521b4295 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:11.288206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 862/1722] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 05:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:59:11.306364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] The token cache is being invalidate because project b30c7d625bdf499b8aae3edbff9926fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:11.306364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:11.306364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66243aa9-81c3-4144-8067-0d7d8f1255fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:11.314165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 861/1723] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:10 2022] DELETE /identity/v3/projects/b30c7d625bdf499b8aae3edbff9926fa => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:11.319050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773c8e8-e489-4314-b0f2-1cc394b8ce5a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:11.319505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773c8e8-e489-4314-b0f2-1cc394b8ce5a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:11.319946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773c8e8-e489-4314-b0f2-1cc394b8ce5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:11.349617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e773c8e8-e489-4314-b0f2-1cc394b8ce5a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:11.356509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e773c8e8-e489-4314-b0f2-1cc394b8ce5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:11.396156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:11.501847 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99816b37-db20-4c82-a7d6-406d0efb51ce None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:11.502293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99816b37-db20-4c82-a7d6-406d0efb51ce None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:11.502647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99816b37-db20-4c82-a7d6-406d0efb51ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:11.510054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 863/1724] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:11.511690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32db691-6bf8-49e1-9051-0874f8ac6281 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:11.512046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32db691-6bf8-49e1-9051-0874f8ac6281 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:11.512403 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32db691-6bf8-49e1-9051-0874f8ac6281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:11.535529 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-b32db691-6bf8-49e1-9051-0874f8ac6281 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:11.537364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b32db691-6bf8-49e1-9051-0874f8ac6281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:11.568225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 864/1725] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:11.569196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32db691-6bf8-49e1-9051-0874f8ac6281 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:11.584276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-99816b37-db20-4c82-a7d6-406d0efb51ce None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:11.585719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-99816b37-db20-4c82-a7d6-406d0efb51ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:11.596417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32db691-6bf8-49e1-9051-0874f8ac6281 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:11.607822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a085059-85d9-45f2-8870-8ab5a615f908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:11.608548 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a085059-85d9-45f2-8870-8ab5a615f908 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:11.608834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a085059-85d9-45f2-8870-8ab5a615f908 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:11.609655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a085059-85d9-45f2-8870-8ab5a615f908 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:11.612477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a085059-85d9-45f2-8870-8ab5a615f908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:11.612797 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a085059-85d9-45f2-8870-8ab5a615f908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:11.614521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a085059-85d9-45f2-8870-8ab5a615f908 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:11.651741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 865/1726] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:11.657383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a085059-85d9-45f2-8870-8ab5a615f908 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:11.686743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a085059-85d9-45f2-8870-8ab5a615f908 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:11.715259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbc518-4862-439e-9610-823b5ac98da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:11.716136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbc518-4862-439e-9610-823b5ac98da3 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:11.716433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbc518-4862-439e-9610-823b5ac98da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:11.719647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbc518-4862-439e-9610-823b5ac98da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:11.719647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbc518-4862-439e-9610-823b5ac98da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:11.727985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbc518-4862-439e-9610-823b5ac98da3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:11.810668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 862/1727] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 05:59:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:11.839330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 866/1728] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:11 2022] POST /identity/v3/projects => generated 363 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:11.851956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99816b37-db20-4c82-a7d6-406d0efb51ce None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:11.866664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99816b37-db20-4c82-a7d6-406d0efb51ce None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:11.895748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbc518-4862-439e-9610-823b5ac98da3 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:11.898118 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-61fbc518-4862-439e-9610-823b5ac98da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:11.961207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162df95-6c22-4dc2-a3f7-9a22a4249a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:11.961207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162df95-6c22-4dc2-a3f7-9a22a4249a7d admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:11.961207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162df95-6c22-4dc2-a3f7-9a22a4249a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:11.961207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162df95-6c22-4dc2-a3f7-9a22a4249a7d admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:11.961605 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162df95-6c22-4dc2-a3f7-9a22a4249a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:11.963558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162df95-6c22-4dc2-a3f7-9a22a4249a7d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:11.999410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbc518-4862-439e-9610-823b5ac98da3 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:12.011059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f4f9e-e1c7-498c-8c51-75643236c6eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:12.011954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f4f9e-e1c7-498c-8c51-75643236c6eb service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:12.012438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f4f9e-e1c7-498c-8c51-75643236c6eb service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:12.012680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f4f9e-e1c7-498c-8c51-75643236c6eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:12.024121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f4f9e-e1c7-498c-8c51-75643236c6eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:12.025756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f4f9e-e1c7-498c-8c51-75643236c6eb service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:12.038985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 867/1729] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:12.077090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 863/1730] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:11 2022] POST /identity/v3/users => generated 282 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:12.079306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f4f9e-e1c7-498c-8c51-75643236c6eb service nova] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:12.107983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f4f9e-e1c7-498c-8c51-75643236c6eb service nova] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:12.122065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b5b8c-0d05-43f3-944c-42f15c42f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:12.123068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5b8c-0d05-43f3-944c-42f15c42f486 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:12.123607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5b8c-0d05-43f3-944c-42f15c42f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:12.123982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5b8c-0d05-43f3-944c-42f15c42f486 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:12.124864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b5b8c-0d05-43f3-944c-42f15c42f486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:12.126721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b5b8c-0d05-43f3-944c-42f15c42f486 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:12.153316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 868/1731] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:12.154823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162df95-6c22-4dc2-a3f7-9a22a4249a7d admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:12.163183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:12.170006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162df95-6c22-4dc2-a3f7-9a22a4249a7d admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:12.180929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75575bc3-8113-48a6-b19d-dcccc4c10b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:12.181893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75575bc3-8113-48a6-b19d-dcccc4c10b36 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:12.183645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75575bc3-8113-48a6-b19d-dcccc4c10b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:12.183948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75575bc3-8113-48a6-b19d-dcccc4c10b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:12.184715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75575bc3-8113-48a6-b19d-dcccc4c10b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:12.192235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75575bc3-8113-48a6-b19d-dcccc4c10b36 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:12.220107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b5b8c-0d05-43f3-944c-42f15c42f486 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:12.221386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 864/1732] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:12.233809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b5b8c-0d05-43f3-944c-42f15c42f486 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:12.252152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0810-7a82-4b62-831c-52e74f8bb6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:12.253311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0810-7a82-4b62-831c-52e74f8bb6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:12.253780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0810-7a82-4b62-831c-52e74f8bb6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:12.254252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0810-7a82-4b62-831c-52e74f8bb6f2 admin admin] PATH_INFO: `/v3/projects/01d6f7901bc14463a8e5cd8d6c98015a/users/6ad60b023bcf451893cbb5acd88b0589/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:12.320989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb0810-7a82-4b62-831c-52e74f8bb6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d6f7901bc14463a8e5cd8d6c98015a, user_id=6ad60b023bcf451893cbb5acd88b0589, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:12.327546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb0810-7a82-4b62-831c-52e74f8bb6f2 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:12.403099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 869/1733] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:12 2022] PUT /identity/v3/projects/01d6f7901bc14463a8e5cd8d6c98015a/users/6ad60b023bcf451893cbb5acd88b0589/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:12.418068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f298e6-0d4f-4576-bda3-3d16811b4edb None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:12.418901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f298e6-0d4f-4576-bda3-3d16811b4edb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:12.419512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f298e6-0d4f-4576-bda3-3d16811b4edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:12.475940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-43f298e6-0d4f-4576-bda3-3d16811b4edb None None] MFA Rules not processed for user `6ad60b023bcf451893cbb5acd88b0589`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:12.477418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-43f298e6-0d4f-4576-bda3-3d16811b4edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:12.573148 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 865/1734] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:12 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:12.577736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0810-7a82-4b62-831c-52e74f8bb6f2 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:12.619336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:12.654203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0810-7a82-4b62-831c-52e74f8bb6f2 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:12.675321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4c5ab-f553-44f3-801d-479e52db2ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:12.676045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4c5ab-f553-44f3-801d-479e52db2ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:12.676280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4c5ab-f553-44f3-801d-479e52db2ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:12.676560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4c5ab-f553-44f3-801d-479e52db2ea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:12.677323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4c5ab-f553-44f3-801d-479e52db2ea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:12.679593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4c5ab-f553-44f3-801d-479e52db2ea3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:12.822805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 870/1735] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:12 2022] POST /identity/v3/projects => generated 363 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:12.828924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f298e6-0d4f-4576-bda3-3d16811b4edb None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:12.841230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f298e6-0d4f-4576-bda3-3d16811b4edb None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:12.897237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cd5dc-86e4-4cae-aacb-0b2968c40c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:12.897914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cd5dc-86e4-4cae-aacb-0b2968c40c09 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:12.898196 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cd5dc-86e4-4cae-aacb-0b2968c40c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:12.898487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cd5dc-86e4-4cae-aacb-0b2968c40c09 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:12.899189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cd5dc-86e4-4cae-aacb-0b2968c40c09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:12.901185 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cd5dc-86e4-4cae-aacb-0b2968c40c09 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:12.991684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 866/1736] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:12 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:12.994950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4c5ab-f553-44f3-801d-479e52db2ea3 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:13.006526 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4c5ab-f553-44f3-801d-479e52db2ea3 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:13.017906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dd5fe-c3bf-4c8f-98ca-2dc0e4fc276a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:13.018700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd5fe-c3bf-4c8f-98ca-2dc0e4fc276a admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:13.019047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd5fe-c3bf-4c8f-98ca-2dc0e4fc276a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:13.019398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd5fe-c3bf-4c8f-98ca-2dc0e4fc276a admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:13.020220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3dd5fe-c3bf-4c8f-98ca-2dc0e4fc276a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:13.021949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3dd5fe-c3bf-4c8f-98ca-2dc0e4fc276a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:13.035205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 871/1737] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:13.039628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cd5dc-86e4-4cae-aacb-0b2968c40c09 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:13.049674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cd5dc-86e4-4cae-aacb-0b2968c40c09 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:13.054254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe1d5c-1f8f-4937-8fbb-b595386de029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:13.055242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe1d5c-1f8f-4937-8fbb-b595386de029 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:13.055595 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe1d5c-1f8f-4937-8fbb-b595386de029 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:13.055941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe1d5c-1f8f-4937-8fbb-b595386de029 admin admin] PATH_INFO: `/v3/projects/4f38195e73ae49d1b5cd469dc6895ffb/users/dcb87780224c4af5a8d87d8da9175105/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:13.070578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe1d5c-1f8f-4937-8fbb-b595386de029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f38195e73ae49d1b5cd469dc6895ffb, user_id=dcb87780224c4af5a8d87d8da9175105, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:13.073231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe1d5c-1f8f-4937-8fbb-b595386de029 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:13.129092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 867/1738] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:13 2022] PUT /identity/v3/projects/4f38195e73ae49d1b5cd469dc6895ffb/users/dcb87780224c4af5a8d87d8da9175105/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:13.133217 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dd5fe-c3bf-4c8f-98ca-2dc0e4fc276a admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:13.147973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dd5fe-c3bf-4c8f-98ca-2dc0e4fc276a admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:13.211886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eab0b7-bbdf-4d56-a31b-7b4876e834ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:13.212513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab0b7-bbdf-4d56-a31b-7b4876e834ad admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:13.212798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab0b7-bbdf-4d56-a31b-7b4876e834ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:13.213077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab0b7-bbdf-4d56-a31b-7b4876e834ad admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:13.213812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eab0b7-bbdf-4d56-a31b-7b4876e834ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:13.215531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eab0b7-bbdf-4d56-a31b-7b4876e834ad admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:13.243402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 872/1739] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:13.248341 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe1d5c-1f8f-4937-8fbb-b595386de029 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:13.260974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe1d5c-1f8f-4937-8fbb-b595386de029 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:13.268108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c6359-4d91-473d-8278-54a259e0e78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:13.269599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c6359-4d91-473d-8278-54a259e0e78f admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:13.270228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c6359-4d91-473d-8278-54a259e0e78f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:13.270228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c6359-4d91-473d-8278-54a259e0e78f admin admin] PATH_INFO: `/v3/projects/4f38195e73ae49d1b5cd469dc6895ffb/users/dcb87780224c4af5a8d87d8da9175105/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:13.272713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c6359-4d91-473d-8278-54a259e0e78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f38195e73ae49d1b5cd469dc6895ffb, user_id=dcb87780224c4af5a8d87d8da9175105, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:13.275964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c6359-4d91-473d-8278-54a259e0e78f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:13.298146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 868/1740] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:13 2022] PUT /identity/v3/projects/4f38195e73ae49d1b5cd469dc6895ffb/users/dcb87780224c4af5a8d87d8da9175105/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:13.309555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041fec8-a7ee-4f6f-98cb-9c22b4ac252d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:13.309951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041fec8-a7ee-4f6f-98cb-9c22b4ac252d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:13.310232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041fec8-a7ee-4f6f-98cb-9c22b4ac252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:13.343921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2041fec8-a7ee-4f6f-98cb-9c22b4ac252d None None] MFA Rules not processed for user `dcb87780224c4af5a8d87d8da9175105`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:13.345271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2041fec8-a7ee-4f6f-98cb-9c22b4ac252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:13.435647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 873/1741] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:13 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:13.451192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c6359-4d91-473d-8278-54a259e0e78f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:13.455174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-308c6359-4d91-473d-8278-54a259e0e78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:13.544655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:13.569109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c6359-4d91-473d-8278-54a259e0e78f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:13.572982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484217a-1603-4526-8d7f-3b2a95676604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:13.573660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484217a-1603-4526-8d7f-3b2a95676604 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:13.573945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484217a-1603-4526-8d7f-3b2a95676604 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:13.574370 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484217a-1603-4526-8d7f-3b2a95676604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:13.594324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484217a-1603-4526-8d7f-3b2a95676604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:13.595858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484217a-1603-4526-8d7f-3b2a95676604 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:13.614714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 869/1742] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:13.910048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041fec8-a7ee-4f6f-98cb-9c22b4ac252d None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:13.931871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041fec8-a7ee-4f6f-98cb-9c22b4ac252d None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:13.935518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed226a-b6ba-4c2c-b9ff-93e71b1b2327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:13.936212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed226a-b6ba-4c2c-b9ff-93e71b1b2327 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:13.936557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed226a-b6ba-4c2c-b9ff-93e71b1b2327 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:13.936835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed226a-b6ba-4c2c-b9ff-93e71b1b2327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:13.966851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:14.018637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed226a-b6ba-4c2c-b9ff-93e71b1b2327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:14.018637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed226a-b6ba-4c2c-b9ff-93e71b1b2327 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:14.028408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 874/1743] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:18.918129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484217a-1603-4526-8d7f-3b2a95676604 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:18.919662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1484217a-1603-4526-8d7f-3b2a95676604 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.050670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484217a-1603-4526-8d7f-3b2a95676604 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.054228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.055282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.056090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.056595 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] PATH_INFO: `/v3/users/64c3a48be83048b2b6dfd91f284bce57` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.058146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c3a48be83048b2b6dfd91f284bce57)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.060667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.294939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c3a48be83048b2b6dfd91f284bce57'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:19.310376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c3a48be83048b2b6dfd91f284bce57'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:19.377671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c3a48be83048b2b6dfd91f284bce57'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:19.378882 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c3a48be83048b2b6dfd91f284bce57'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:19.379570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c3a48be83048b2b6dfd91f284bce57'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:19.435361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 870/1744] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:18 2022] DELETE /identity/v3/users/64c3a48be83048b2b6dfd91f284bce57 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:19.440026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c37c56-10d3-49ec-acbd-9f5a9fe81c2a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.440599 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c37c56-10d3-49ec-acbd-9f5a9fe81c2a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.440689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c37c56-10d3-49ec-acbd-9f5a9fe81c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.461184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-05c37c56-10d3-49ec-acbd-9f5a9fe81c2a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.469227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-05c37c56-10d3-49ec-acbd-9f5a9fe81c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.539133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 875/1745] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:19.561884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.563760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.685579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbeb0f6-d9f8-41ca-a56e-1b7b6ef0b931 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.689415 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7dba36-5c74-42a7-826f-8e2c7e1885c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.690289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dba36-5c74-42a7-826f-8e2c7e1885c5 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.690673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dba36-5c74-42a7-826f-8e2c7e1885c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.696973 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dba36-5c74-42a7-826f-8e2c7e1885c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.702031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7dba36-5c74-42a7-826f-8e2c7e1885c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.704285 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7dba36-5c74-42a7-826f-8e2c7e1885c5 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.727001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:19.729272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 871/1746] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:20.436419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c37c56-10d3-49ec-acbd-9f5a9fe81c2a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.437866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-05c37c56-10d3-49ec-acbd-9f5a9fe81c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:20.450569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c37c56-10d3-49ec-acbd-9f5a9fe81c2a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.457012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.461863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.461863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.461863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] PATH_INFO: `/v3/projects/76b1bb5d3de346669eeaf8767705e4e9` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.461863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b1bb5d3de346669eeaf8767705e4e9)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.464163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.582249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b1bb5d3de346669eeaf8767705e4e9', 'request_id': 'req-88230ca2-3823-410d-8f22-08681bfa25e8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:20.589505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b1bb5d3de346669eeaf8767705e4e9', 'request_id': 'req-88230ca2-3823-410d-8f22-08681bfa25e8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:20.596655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b1bb5d3de346669eeaf8767705e4e9', 'request_id': 'req-88230ca2-3823-410d-8f22-08681bfa25e8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:20.653157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] The token cache is being invalidate because project 76b1bb5d3de346669eeaf8767705e4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:20.653157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:20.653965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-88230ca2-3823-410d-8f22-08681bfa25e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:20.656408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 876/1747] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:20 2022] DELETE /identity/v3/projects/76b1bb5d3de346669eeaf8767705e4e9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:20.661373 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e853c4-9baa-4fb6-a9d5-2348a251ef51 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.661857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e853c4-9baa-4fb6-a9d5-2348a251ef51 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.661962 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e853c4-9baa-4fb6-a9d5-2348a251ef51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.685905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f2e853c4-9baa-4fb6-a9d5-2348a251ef51 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:20.687078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f2e853c4-9baa-4fb6-a9d5-2348a251ef51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:20.781345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 872/1748] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:20.789406 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0ca80-3e34-4643-84ca-39a58fe959bb None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.789697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0ca80-3e34-4643-84ca-39a58fe959bb None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.789981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0ca80-3e34-4643-84ca-39a58fe959bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.823477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c5f0ca80-3e34-4643-84ca-39a58fe959bb None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:20.824936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c5f0ca80-3e34-4643-84ca-39a58fe959bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:20.859566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 877/1749] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:20.859960 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e853c4-9baa-4fb6-a9d5-2348a251ef51 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.875325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e853c4-9baa-4fb6-a9d5-2348a251ef51 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.880585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025556d0-80df-42ed-a172-f9426475f8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.881353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025556d0-80df-42ed-a172-f9426475f8fa admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.881653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025556d0-80df-42ed-a172-f9426475f8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.881939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025556d0-80df-42ed-a172-f9426475f8fa admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.882846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025556d0-80df-42ed-a172-f9426475f8fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:20.883163 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025556d0-80df-42ed-a172-f9426475f8fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.884916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025556d0-80df-42ed-a172-f9426475f8fa admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.909700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0ca80-3e34-4643-84ca-39a58fe959bb None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.916905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 873/1750] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:59:20.920909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0ca80-3e34-4643-84ca-39a58fe959bb None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.926382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82818c-737f-4b92-9e64-402604438ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.927407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82818c-737f-4b92-9e64-402604438ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.927823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82818c-737f-4b92-9e64-402604438ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.928241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82818c-737f-4b92-9e64-402604438ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.929022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82818c-737f-4b92-9e64-402604438ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.931372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82818c-737f-4b92-9e64-402604438ed5 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.051954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025556d0-80df-42ed-a172-f9426475f8fa admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.052701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 878/1751] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:21.064055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025556d0-80df-42ed-a172-f9426475f8fa admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.160252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4806e-6b66-4561-a413-db42970ab797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.161137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4806e-6b66-4561-a413-db42970ab797 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.161427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4806e-6b66-4561-a413-db42970ab797 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.161705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4806e-6b66-4561-a413-db42970ab797 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.163385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4806e-6b66-4561-a413-db42970ab797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.168396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4806e-6b66-4561-a413-db42970ab797 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.183901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 874/1752] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:21 2022] POST /identity/v3/users => generated 283 bytes in 1132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:22.189652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82818c-737f-4b92-9e64-402604438ed5 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.216931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82818c-737f-4b92-9e64-402604438ed5 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.222903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9d9fd-f527-4342-988b-433315adc7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.223676 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9d9fd-f527-4342-988b-433315adc7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.224142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9d9fd-f527-4342-988b-433315adc7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.224633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9d9fd-f527-4342-988b-433315adc7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.225427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9d9fd-f527-4342-988b-433315adc7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.227264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9d9fd-f527-4342-988b-433315adc7d0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.239275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 879/1753] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:22.244093 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4806e-6b66-4561-a413-db42970ab797 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.262129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4806e-6b66-4561-a413-db42970ab797 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.271044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10ff10-0129-45eb-9c55-b3b3d0e6e86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.272107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10ff10-0129-45eb-9c55-b3b3d0e6e86e admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.272568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10ff10-0129-45eb-9c55-b3b3d0e6e86e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.273522 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10ff10-0129-45eb-9c55-b3b3d0e6e86e admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.273949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10ff10-0129-45eb-9c55-b3b3d0e6e86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.275699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10ff10-0129-45eb-9c55-b3b3d0e6e86e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.290498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 875/1754] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:22.291747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9d9fd-f527-4342-988b-433315adc7d0 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.299782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:22.303935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9d9fd-f527-4342-988b-433315adc7d0 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.311391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2df73-04ef-4c04-8613-5a81d7c52996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.312425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2df73-04ef-4c04-8613-5a81d7c52996 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.312772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2df73-04ef-4c04-8613-5a81d7c52996 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.313114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2df73-04ef-4c04-8613-5a81d7c52996 admin admin] PATH_INFO: `/v3/projects/72d1271111b24b9c9657bca261c35cac/users/f033c346100d4fa081571f86f6b22e70/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.328226 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2df73-04ef-4c04-8613-5a81d7c52996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d1271111b24b9c9657bca261c35cac, user_id=f033c346100d4fa081571f86f6b22e70, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.330915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2df73-04ef-4c04-8613-5a81d7c52996 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.543095 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 880/1755] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:22 2022] PUT /identity/v3/projects/72d1271111b24b9c9657bca261c35cac/users/f033c346100d4fa081571f86f6b22e70/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:22.554631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e546cd-9f90-49a7-9aa3-2e55402281d0 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.555067 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e546cd-9f90-49a7-9aa3-2e55402281d0 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.555347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e546cd-9f90-49a7-9aa3-2e55402281d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.595317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-25e546cd-9f90-49a7-9aa3-2e55402281d0 None None] MFA Rules not processed for user `f033c346100d4fa081571f86f6b22e70`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:22.599940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-25e546cd-9f90-49a7-9aa3-2e55402281d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:22.685178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:22.696897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 876/1756] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:22.701033 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2df73-04ef-4c04-8613-5a81d7c52996 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.828307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2df73-04ef-4c04-8613-5a81d7c52996 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.833659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1375a8-1803-421f-8ea4-76ecfab9c5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.834231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1375a8-1803-421f-8ea4-76ecfab9c5da admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.834534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1375a8-1803-421f-8ea4-76ecfab9c5da admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.834922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1375a8-1803-421f-8ea4-76ecfab9c5da admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.835631 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1375a8-1803-421f-8ea4-76ecfab9c5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.838313 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1375a8-1803-421f-8ea4-76ecfab9c5da admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.931015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 881/1757] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:22 2022] POST /identity/v3/projects => generated 363 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:22.932409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e546cd-9f90-49a7-9aa3-2e55402281d0 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.942818 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e546cd-9f90-49a7-9aa3-2e55402281d0 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.010640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2cec0-7a89-4c86-a330-7c69510782c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.011842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2cec0-7a89-4c86-a330-7c69510782c2 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.012186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2cec0-7a89-4c86-a330-7c69510782c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.012574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2cec0-7a89-4c86-a330-7c69510782c2 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.013376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2cec0-7a89-4c86-a330-7c69510782c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.016115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2cec0-7a89-4c86-a330-7c69510782c2 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.084154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 877/1758] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:22 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:23.085412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1375a8-1803-421f-8ea4-76ecfab9c5da admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.095806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1375a8-1803-421f-8ea4-76ecfab9c5da admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.101259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487da53-532c-4942-ae6b-bbed667f7c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.103104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487da53-532c-4942-ae6b-bbed667f7c7d admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.103388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487da53-532c-4942-ae6b-bbed667f7c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.103668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487da53-532c-4942-ae6b-bbed667f7c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.104383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487da53-532c-4942-ae6b-bbed667f7c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.105979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487da53-532c-4942-ae6b-bbed667f7c7d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.117332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 882/1759] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:23.121582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2cec0-7a89-4c86-a330-7c69510782c2 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.132260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2cec0-7a89-4c86-a330-7c69510782c2 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.138290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f6e0f-5f6b-4ab8-81e0-5b1c6224ccf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.139218 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f6e0f-5f6b-4ab8-81e0-5b1c6224ccf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.139502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f6e0f-5f6b-4ab8-81e0-5b1c6224ccf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.139776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f6e0f-5f6b-4ab8-81e0-5b1c6224ccf5 admin admin] PATH_INFO: `/v3/projects/cac6fa40a22f4a2f87bafa83b3bcc71d/users/eb3a75c5cc264f98b386ca4edcc72dc9/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.162384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f6e0f-5f6b-4ab8-81e0-5b1c6224ccf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac6fa40a22f4a2f87bafa83b3bcc71d, user_id=eb3a75c5cc264f98b386ca4edcc72dc9, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.162384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f6e0f-5f6b-4ab8-81e0-5b1c6224ccf5 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.240385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 878/1760] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:23 2022] PUT /identity/v3/projects/cac6fa40a22f4a2f87bafa83b3bcc71d/users/eb3a75c5cc264f98b386ca4edcc72dc9/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:23.244299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487da53-532c-4942-ae6b-bbed667f7c7d admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.254879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487da53-532c-4942-ae6b-bbed667f7c7d admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.310251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cd00c-2521-4e42-8730-aff453b1a65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.310980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cd00c-2521-4e42-8730-aff453b1a65b admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.311258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cd00c-2521-4e42-8730-aff453b1a65b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.311531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cd00c-2521-4e42-8730-aff453b1a65b admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.312772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cd00c-2521-4e42-8730-aff453b1a65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.314386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cd00c-2521-4e42-8730-aff453b1a65b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.327507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 883/1761] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:23.329173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f6e0f-5f6b-4ab8-81e0-5b1c6224ccf5 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.339244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f6e0f-5f6b-4ab8-81e0-5b1c6224ccf5 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.346657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a5b91-3897-4e94-80da-454d3d92c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.347698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a5b91-3897-4e94-80da-454d3d92c0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.347985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a5b91-3897-4e94-80da-454d3d92c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.348323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a5b91-3897-4e94-80da-454d3d92c0d2 admin admin] PATH_INFO: `/v3/projects/cac6fa40a22f4a2f87bafa83b3bcc71d/users/eb3a75c5cc264f98b386ca4edcc72dc9/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.350400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a5b91-3897-4e94-80da-454d3d92c0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac6fa40a22f4a2f87bafa83b3bcc71d, user_id=eb3a75c5cc264f98b386ca4edcc72dc9, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.353083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a5b91-3897-4e94-80da-454d3d92c0d2 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.364710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 879/1762] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:23 2022] PUT /identity/v3/projects/cac6fa40a22f4a2f87bafa83b3bcc71d/users/eb3a75c5cc264f98b386ca4edcc72dc9/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:23.376908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedc15b-8c1e-4619-8bf5-11c36cdfe21b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.377216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedc15b-8c1e-4619-8bf5-11c36cdfe21b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.378119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedc15b-8c1e-4619-8bf5-11c36cdfe21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.410262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ffedc15b-8c1e-4619-8bf5-11c36cdfe21b None None] MFA Rules not processed for user `eb3a75c5cc264f98b386ca4edcc72dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:23.411197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ffedc15b-8c1e-4619-8bf5-11c36cdfe21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:23.488890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 884/1763] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:23 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:23.500884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a5b91-3897-4e94-80da-454d3d92c0d2 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.502768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a20a5b91-3897-4e94-80da-454d3d92c0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:23.570819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a5b91-3897-4e94-80da-454d3d92c0d2 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.574386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832401db-6802-4595-8ab8-54317f36d3a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.575111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832401db-6802-4595-8ab8-54317f36d3a2 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.575395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832401db-6802-4595-8ab8-54317f36d3a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.575669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832401db-6802-4595-8ab8-54317f36d3a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.585527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832401db-6802-4595-8ab8-54317f36d3a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.587011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832401db-6802-4595-8ab8-54317f36d3a2 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.598569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 880/1764] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:23 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:23.798284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedc15b-8c1e-4619-8bf5-11c36cdfe21b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.813076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedc15b-8c1e-4619-8bf5-11c36cdfe21b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.817390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9465bae-b9ec-4be8-80a7-2b238bbded7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.818099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9465bae-b9ec-4be8-80a7-2b238bbded7c service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.818334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9465bae-b9ec-4be8-80a7-2b238bbded7c service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.818637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9465bae-b9ec-4be8-80a7-2b238bbded7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.934157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9465bae-b9ec-4be8-80a7-2b238bbded7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.935995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9465bae-b9ec-4be8-80a7-2b238bbded7c service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.949338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 885/1765] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:24.540332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:26.524359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f4700-e66b-42fd-bc11-111e59789da8 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:26.524947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f4700-e66b-42fd-bc11-111e59789da8 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:26.524947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f4700-e66b-42fd-bc11-111e59789da8 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:26.527503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 881/1766] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 05:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:59:26.561168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fc75a-9047-44f7-9e5d-f861e2030ac7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:26.561731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fc75a-9047-44f7-9e5d-f861e2030ac7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:26.564505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fc75a-9047-44f7-9e5d-f861e2030ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:26.593504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-af8fc75a-9047-44f7-9e5d-f861e2030ac7 None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:26.596886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-af8fc75a-9047-44f7-9e5d-f861e2030ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:26.725486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 886/1767] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 05:59:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:26.758613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f4700-e66b-42fd-bc11-111e59789da8 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:26.760380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-073f4700-e66b-42fd-bc11-111e59789da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:26.782249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f4700-e66b-42fd-bc11-111e59789da8 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:26.789399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac67a96-8ab9-4619-9e6e-eb0d6a996f10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:26.790053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac67a96-8ab9-4619-9e6e-eb0d6a996f10 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:26.790693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac67a96-8ab9-4619-9e6e-eb0d6a996f10 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:26.790967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac67a96-8ab9-4619-9e6e-eb0d6a996f10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:26.805080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac67a96-8ab9-4619-9e6e-eb0d6a996f10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:26.806894 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac67a96-8ab9-4619-9e6e-eb0d6a996f10 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:26.818893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 882/1768] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:28.009153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fc75a-9047-44f7-9e5d-f861e2030ac7 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:28.088862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fc75a-9047-44f7-9e5d-f861e2030ac7 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:28.093803 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:28.094846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.095231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.095633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] PATH_INFO: `/v3/users/f033c346100d4fa081571f86f6b22e70` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.097324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f033c346100d4fa081571f86f6b22e70)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:28.100253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:28.107619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:28.222846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac67a96-8ab9-4619-9e6e-eb0d6a996f10 service nova] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:28.225371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-eac67a96-8ab9-4619-9e6e-eb0d6a996f10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:28.239643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac67a96-8ab9-4619-9e6e-eb0d6a996f10 service nova] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:28.244077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:28.246692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.246692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.246692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] PATH_INFO: `/v3/users/6ad60b023bcf451893cbb5acd88b0589` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.246692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad60b023bcf451893cbb5acd88b0589)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:28.248555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:28.271987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f033c346100d4fa081571f86f6b22e70'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:28.278873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f033c346100d4fa081571f86f6b22e70'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:28.279538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f033c346100d4fa081571f86f6b22e70'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:28.354576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f033c346100d4fa081571f86f6b22e70'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:28.355205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5376ff3d-5e4d-4b9f-9590-9d6de96b386a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f033c346100d4fa081571f86f6b22e70'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:28.373449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 887/1769] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:28 2022] DELETE /identity/v3/users/f033c346100d4fa081571f86f6b22e70 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:28.383253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e6cd5-044a-43dd-8f5c-5acd85103df6 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.383544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e6cd5-044a-43dd-8f5c-5acd85103df6 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.383827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e6cd5-044a-43dd-8f5c-5acd85103df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.400793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-369e6cd5-044a-43dd-8f5c-5acd85103df6 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:28.401999 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-369e6cd5-044a-43dd-8f5c-5acd85103df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:28.425270 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:28.437336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad60b023bcf451893cbb5acd88b0589'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:28.444907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad60b023bcf451893cbb5acd88b0589'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:28.463245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 888/1770] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:28.473707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e6cd5-044a-43dd-8f5c-5acd85103df6 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:28.475154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-369e6cd5-044a-43dd-8f5c-5acd85103df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:28.520986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad60b023bcf451893cbb5acd88b0589'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:28.522030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad60b023bcf451893cbb5acd88b0589'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:28.523278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2334e141-33ad-47ee-b2d7-12ee00af0697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad60b023bcf451893cbb5acd88b0589'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:28.540402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e6cd5-044a-43dd-8f5c-5acd85103df6 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:28.545736 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 883/1771] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:28 2022] DELETE /identity/v3/users/6ad60b023bcf451893cbb5acd88b0589 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:28.549785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffadcc-c93c-4cd9-9d0e-7123d71fbfbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:28.550501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffadcc-c93c-4cd9-9d0e-7123d71fbfbe service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.551867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06620d-8820-4fad-a03b-6eb3151604c2 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.552233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06620d-8820-4fad-a03b-6eb3151604c2 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.552509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06620d-8820-4fad-a03b-6eb3151604c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.554051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffadcc-c93c-4cd9-9d0e-7123d71fbfbe service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.554328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffadcc-c93c-4cd9-9d0e-7123d71fbfbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.556193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-51ffadcc-c93c-4cd9-9d0e-7123d71fbfbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:28.568691 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffadcc-c93c-4cd9-9d0e-7123d71fbfbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:28.570125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffadcc-c93c-4cd9-9d0e-7123d71fbfbe service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:28.573673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-cb06620d-8820-4fad-a03b-6eb3151604c2 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:28.574933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cb06620d-8820-4fad-a03b-6eb3151604c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:28.581981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 889/1772] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:28.651101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 884/1773] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:28.662726 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffadcc-c93c-4cd9-9d0e-7123d71fbfbe service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:28.664186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-51ffadcc-c93c-4cd9-9d0e-7123d71fbfbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:28.679842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffadcc-c93c-4cd9-9d0e-7123d71fbfbe service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:28.708480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:28.733071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab844b-5b92-45b3-8be9-8fe087908d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:28.736839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab844b-5b92-45b3-8be9-8fe087908d8a service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.736839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab844b-5b92-45b3-8be9-8fe087908d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.736839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab844b-5b92-45b3-8be9-8fe087908d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.744122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab844b-5b92-45b3-8be9-8fe087908d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:28.745563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab844b-5b92-45b3-8be9-8fe087908d8a service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:28.762147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 890/1774] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:29.157530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06620d-8820-4fad-a03b-6eb3151604c2 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:29.159803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cb06620d-8820-4fad-a03b-6eb3151604c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:29.175472 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06620d-8820-4fad-a03b-6eb3151604c2 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:29.184840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:29.185834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:29.186221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:29.186525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] PATH_INFO: `/v3/projects/72d1271111b24b9c9657bca261c35cac` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:29.189588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d1271111b24b9c9657bca261c35cac)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:29.192335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:29.325245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d1271111b24b9c9657bca261c35cac', 'request_id': 'req-cfa213da-81d2-4852-88d8-174163d1b73f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.336713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d1271111b24b9c9657bca261c35cac', 'request_id': 'req-cfa213da-81d2-4852-88d8-174163d1b73f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.344345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d1271111b24b9c9657bca261c35cac', 'request_id': 'req-cfa213da-81d2-4852-88d8-174163d1b73f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.434764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] The token cache is being invalidate because project 72d1271111b24b9c9657bca261c35cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:29.435051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.435873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.438090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 885/1775] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:29 2022] DELETE /identity/v3/projects/72d1271111b24b9c9657bca261c35cac => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:29.442137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab844b-5b92-45b3-8be9-8fe087908d8a service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:29.452144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab844b-5b92-45b3-8be9-8fe087908d8a service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:29.499447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:29.501519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:29.512680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:29.523802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:29.524019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:29.524294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] PATH_INFO: `/v3/users/eb3a75c5cc264f98b386ca4edcc72dc9` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:29.525653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb3a75c5cc264f98b386ca4edcc72dc9)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:29.527659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:29.547109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:29.578162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa213da-81d2-4852-88d8-174163d1b73f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:29.583041 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:29.583907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:29.584190 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:29.584486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] PATH_INFO: `/v3/projects/01d6f7901bc14463a8e5cd8d6c98015a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:29.585664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d6f7901bc14463a8e5cd8d6c98015a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:29.587620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:29.694853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb3a75c5cc264f98b386ca4edcc72dc9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.701812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb3a75c5cc264f98b386ca4edcc72dc9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.702577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb3a75c5cc264f98b386ca4edcc72dc9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.709832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d6f7901bc14463a8e5cd8d6c98015a', 'request_id': 'req-a3b9beec-56a4-4301-a893-f20ed7c0933b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.716165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d6f7901bc14463a8e5cd8d6c98015a', 'request_id': 'req-a3b9beec-56a4-4301-a893-f20ed7c0933b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.722122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d6f7901bc14463a8e5cd8d6c98015a', 'request_id': 'req-a3b9beec-56a4-4301-a893-f20ed7c0933b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.800251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb3a75c5cc264f98b386ca4edcc72dc9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.800822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb3a75c5cc264f98b386ca4edcc72dc9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.818708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 891/1776] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:29 2022] DELETE /identity/v3/users/eb3a75c5cc264f98b386ca4edcc72dc9 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:29.868701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] The token cache is being invalidate because project 01d6f7901bc14463a8e5cd8d6c98015a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:29.868989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.869848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:29.871447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 886/1777] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:29 2022] DELETE /identity/v3/projects/01d6f7901bc14463a8e5cd8d6c98015a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:29.875597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:29.877055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:29.948152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:29.960827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5046e3-cc4f-49aa-aff9-37f2fa7ef0d1 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:29.965822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:29.966764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:29.967047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:29.967319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] PATH_INFO: `/v3/users/dcb87780224c4af5a8d87d8da9175105` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:29.969159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb87780224c4af5a8d87d8da9175105)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:29.971170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:30.176965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb87780224c4af5a8d87d8da9175105'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:30.184695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb87780224c4af5a8d87d8da9175105'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:30.185832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb87780224c4af5a8d87d8da9175105'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:30.262721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb87780224c4af5a8d87d8da9175105'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:30.264493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f482559-6327-45f2-bc2f-4631e06c8ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb87780224c4af5a8d87d8da9175105'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:30.293963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 892/1778] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:29 2022] DELETE /identity/v3/users/dcb87780224c4af5a8d87d8da9175105 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:30.375545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:30.376840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:30.483632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9beec-56a4-4301-a893-f20ed7c0933b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:30.554380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:30.555299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:30.555578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:30.555851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] PATH_INFO: `/v3/projects/cac6fa40a22f4a2f87bafa83b3bcc71d` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:30.557056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cac6fa40a22f4a2f87bafa83b3bcc71d)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:30.560090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:30.603921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:30.722397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac6fa40a22f4a2f87bafa83b3bcc71d', 'request_id': 'req-34e8a621-a8c7-4324-8d6a-01ae7decf002'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:30.730217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac6fa40a22f4a2f87bafa83b3bcc71d', 'request_id': 'req-34e8a621-a8c7-4324-8d6a-01ae7decf002'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:30.740010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac6fa40a22f4a2f87bafa83b3bcc71d', 'request_id': 'req-34e8a621-a8c7-4324-8d6a-01ae7decf002'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:30.857235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] The token cache is being invalidate because project cac6fa40a22f4a2f87bafa83b3bcc71d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:30.857631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:30.858314 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:30.860507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 887/1779] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:30 2022] DELETE /identity/v3/projects/cac6fa40a22f4a2f87bafa83b3bcc71d => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:30.868785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417c73a-f8a5-428f-8d8e-611539267ae7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:30.869276 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417c73a-f8a5-428f-8d8e-611539267ae7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:30.869426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417c73a-f8a5-428f-8d8e-611539267ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:30.886418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:30.887886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:30.909345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-6417c73a-f8a5-428f-8d8e-611539267ae7 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:30.911947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6417c73a-f8a5-428f-8d8e-611539267ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:30.983305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a621-a8c7-4324-8d6a-01ae7decf002 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:30.992596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:30.993777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:30.993977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:30.994250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] PATH_INFO: `/v3/projects/4f38195e73ae49d1b5cd469dc6895ffb` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:30.995851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f38195e73ae49d1b5cd469dc6895ffb)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:30.997978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.058991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 893/1780] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:31.068313 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f43411-c2c4-4431-a8c4-6792278bb6bf None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.068607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f43411-c2c4-4431-a8c4-6792278bb6bf None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.068890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f43411-c2c4-4431-a8c4-6792278bb6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.118853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-f7f43411-c2c4-4431-a8c4-6792278bb6bf None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:31.123112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f7f43411-c2c4-4431-a8c4-6792278bb6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:31.144159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f38195e73ae49d1b5cd469dc6895ffb', 'request_id': 'req-9c22b64e-f59f-4762-ae10-3027e8225b54'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:31.161171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f38195e73ae49d1b5cd469dc6895ffb', 'request_id': 'req-9c22b64e-f59f-4762-ae10-3027e8225b54'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:31.189925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f38195e73ae49d1b5cd469dc6895ffb', 'request_id': 'req-9c22b64e-f59f-4762-ae10-3027e8225b54'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:31.210630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 894/1781] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:31.213878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f43411-c2c4-4431-a8c4-6792278bb6bf None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.224109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f43411-c2c4-4431-a8c4-6792278bb6bf None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.231959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379dca3-7ee1-4209-9439-aed4749fb25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:31.232645 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379dca3-7ee1-4209-9439-aed4749fb25f admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.233806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379dca3-7ee1-4209-9439-aed4749fb25f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.233806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379dca3-7ee1-4209-9439-aed4749fb25f admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.234669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379dca3-7ee1-4209-9439-aed4749fb25f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:31.234824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379dca3-7ee1-4209-9439-aed4749fb25f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:31.236503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379dca3-7ee1-4209-9439-aed4749fb25f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.244435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:31.249834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 895/1782] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:31.254024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379dca3-7ee1-4209-9439-aed4749fb25f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.264461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379dca3-7ee1-4209-9439-aed4749fb25f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.278418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ca704-4093-49ee-ad9d-d1f0a3003810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:31.279055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ca704-4093-49ee-ad9d-d1f0a3003810 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.279340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ca704-4093-49ee-ad9d-d1f0a3003810 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.279618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ca704-4093-49ee-ad9d-d1f0a3003810 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.280382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ca704-4093-49ee-ad9d-d1f0a3003810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:31.284031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ca704-4093-49ee-ad9d-d1f0a3003810 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.367246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] The token cache is being invalidate because project 4f38195e73ae49d1b5cd469dc6895ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:31.367630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:31.368397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c22b64e-f59f-4762-ae10-3027e8225b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:31.371069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 888/1783] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:30 2022] DELETE /identity/v3/projects/4f38195e73ae49d1b5cd469dc6895ffb => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:31.375753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff7b94-14bb-492c-8bef-fc643f25f4d4 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.376334 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff7b94-14bb-492c-8bef-fc643f25f4d4 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.376425 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff7b94-14bb-492c-8bef-fc643f25f4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.393230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 896/1784] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:31 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:59:31.397331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ca704-4093-49ee-ad9d-d1f0a3003810 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.399109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ab1ca704-4093-49ee-ad9d-d1f0a3003810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:31.412888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a0ff7b94-14bb-492c-8bef-fc643f25f4d4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:31.414534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a0ff7b94-14bb-492c-8bef-fc643f25f4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:31.473494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:31.503307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ca704-4093-49ee-ad9d-d1f0a3003810 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.511968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8941701-c0c5-4fcd-a03c-dcfe11091660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:31.512841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8941701-c0c5-4fcd-a03c-dcfe11091660 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.513653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8941701-c0c5-4fcd-a03c-dcfe11091660 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.514505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8941701-c0c5-4fcd-a03c-dcfe11091660 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.515532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8941701-c0c5-4fcd-a03c-dcfe11091660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:31.521468 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8941701-c0c5-4fcd-a03c-dcfe11091660 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.544069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 889/1785] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:31.553880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad25621-8c21-42a1-9a04-e2a8212c04ac None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.554239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad25621-8c21-42a1-9a04-e2a8212c04ac None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.554619 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad25621-8c21-42a1-9a04-e2a8212c04ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.577604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-fad25621-8c21-42a1-9a04-e2a8212c04ac None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:31.579042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-fad25621-8c21-42a1-9a04-e2a8212c04ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:31.587388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 890/1786] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:31.592389 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad25621-8c21-42a1-9a04-e2a8212c04ac None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.605411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad25621-8c21-42a1-9a04-e2a8212c04ac None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.612508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c50bf-f757-4a7c-8eca-e4ffb5ac46fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:31.613946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c50bf-f757-4a7c-8eca-e4ffb5ac46fe admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.614417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c50bf-f757-4a7c-8eca-e4ffb5ac46fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.614841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c50bf-f757-4a7c-8eca-e4ffb5ac46fe admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.615690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c50bf-f757-4a7c-8eca-e4ffb5ac46fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:31.616099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c50bf-f757-4a7c-8eca-e4ffb5ac46fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:31.617462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 897/1787] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:31 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:59:31.618232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c50bf-f757-4a7c-8eca-e4ffb5ac46fe admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.618466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8941701-c0c5-4fcd-a03c-dcfe11091660 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.632505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 891/1788] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:59:31.635751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8941701-c0c5-4fcd-a03c-dcfe11091660 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.636832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c50bf-f757-4a7c-8eca-e4ffb5ac46fe admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.644903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec942b65-2fbf-4f1f-80b1-c30e27db6294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:31.644903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec942b65-2fbf-4f1f-80b1-c30e27db6294 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.644903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec942b65-2fbf-4f1f-80b1-c30e27db6294 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.644903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec942b65-2fbf-4f1f-80b1-c30e27db6294 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.645225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec942b65-2fbf-4f1f-80b1-c30e27db6294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:31.648424 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec942b65-2fbf-4f1f-80b1-c30e27db6294 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.650358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c50bf-f757-4a7c-8eca-e4ffb5ac46fe admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.655561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a3f4-69af-48bc-9e55-5649d639b09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:31.657263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a3f4-69af-48bc-9e55-5649d639b09b admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.657609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a3f4-69af-48bc-9e55-5649d639b09b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.657892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a3f4-69af-48bc-9e55-5649d639b09b admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.658624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a3f4-69af-48bc-9e55-5649d639b09b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:31.660591 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a3f4-69af-48bc-9e55-5649d639b09b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.665298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 898/1789] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:31.669274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec942b65-2fbf-4f1f-80b1-c30e27db6294 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.683201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec942b65-2fbf-4f1f-80b1-c30e27db6294 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.688829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725a36e-74aa-4afd-8eb4-e15785e7e9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:31.689516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725a36e-74aa-4afd-8eb4-e15785e7e9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.689795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725a36e-74aa-4afd-8eb4-e15785e7e9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.690070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725a36e-74aa-4afd-8eb4-e15785e7e9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.690841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725a36e-74aa-4afd-8eb4-e15785e7e9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:31.692492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725a36e-74aa-4afd-8eb4-e15785e7e9e9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.704466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 899/1790] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:31.723895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725a36e-74aa-4afd-8eb4-e15785e7e9e9 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.735144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725a36e-74aa-4afd-8eb4-e15785e7e9e9 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.740008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec65db7-f09b-4bb0-97c1-e5c58a650994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:31.740895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec65db7-f09b-4bb0-97c1-e5c58a650994 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.741179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec65db7-f09b-4bb0-97c1-e5c58a650994 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.741456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec65db7-f09b-4bb0-97c1-e5c58a650994 admin admin] PATH_INFO: `/v3/projects/c2f15693b89b47c4a72fe5485d5afb65/users/ee63049c8c7b42a8a77e3728d768e5dc/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.756156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec65db7-f09b-4bb0-97c1-e5c58a650994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f15693b89b47c4a72fe5485d5afb65, user_id=ee63049c8c7b42a8a77e3728d768e5dc, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:31.758979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec65db7-f09b-4bb0-97c1-e5c58a650994 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.768923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 892/1791] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:31 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:31.773026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a3f4-69af-48bc-9e55-5649d639b09b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.836177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a3f4-69af-48bc-9e55-5649d639b09b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.841225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d498082-8a42-49f8-9eaf-f59d2ab7c6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:31.841900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d498082-8a42-49f8-9eaf-f59d2ab7c6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.842186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d498082-8a42-49f8-9eaf-f59d2ab7c6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.842466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d498082-8a42-49f8-9eaf-f59d2ab7c6b2 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.843199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d498082-8a42-49f8-9eaf-f59d2ab7c6b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:31.845835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d498082-8a42-49f8-9eaf-f59d2ab7c6b2 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.869376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 900/1792] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:31 2022] PUT /identity/v3/projects/c2f15693b89b47c4a72fe5485d5afb65/users/ee63049c8c7b42a8a77e3728d768e5dc/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:31.879431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8dde2-1295-4085-826d-e148a4cd48d9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.879657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8dde2-1295-4085-826d-e148a4cd48d9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.879944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8dde2-1295-4085-826d-e148a4cd48d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.906494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-21c8dde2-1295-4085-826d-e148a4cd48d9 None None] MFA Rules not processed for user `ee63049c8c7b42a8a77e3728d768e5dc`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:31.907791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-21c8dde2-1295-4085-826d-e148a4cd48d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:31.948799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 893/1793] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:31 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:59:31.953350 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d498082-8a42-49f8-9eaf-f59d2ab7c6b2 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.962734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d498082-8a42-49f8-9eaf-f59d2ab7c6b2 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.973059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 901/1794] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:31.980112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8dde2-1295-4085-826d-e148a4cd48d9 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.019349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907c1ab-3509-493d-8642-8a704ebb40aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.020024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907c1ab-3509-493d-8642-8a704ebb40aa admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.020301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907c1ab-3509-493d-8642-8a704ebb40aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.020573 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907c1ab-3509-493d-8642-8a704ebb40aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.021301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907c1ab-3509-493d-8642-8a704ebb40aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.022946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907c1ab-3509-493d-8642-8a704ebb40aa admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.033243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 894/1795] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:32.037429 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907c1ab-3509-493d-8642-8a704ebb40aa admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.041250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8dde2-1295-4085-826d-e148a4cd48d9 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.045932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397fe3b-30f0-408a-8f77-8251b2f2b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.046660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397fe3b-30f0-408a-8f77-8251b2f2b62d admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.046948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397fe3b-30f0-408a-8f77-8251b2f2b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.047076 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907c1ab-3509-493d-8642-8a704ebb40aa admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.047230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397fe3b-30f0-408a-8f77-8251b2f2b62d admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.047925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397fe3b-30f0-408a-8f77-8251b2f2b62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.049829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397fe3b-30f0-408a-8f77-8251b2f2b62d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.051971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911fcbc-e060-48de-846c-f5deeece6139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.052748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911fcbc-e060-48de-846c-f5deeece6139 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.053043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911fcbc-e060-48de-846c-f5deeece6139 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.053323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911fcbc-e060-48de-846c-f5deeece6139 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.054054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911fcbc-e060-48de-846c-f5deeece6139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.055677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911fcbc-e060-48de-846c-f5deeece6139 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.066246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 895/1796] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:32.070296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911fcbc-e060-48de-846c-f5deeece6139 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.080239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911fcbc-e060-48de-846c-f5deeece6139 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.080362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:32.084792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59b2ac-77ec-4c42-9cf1-dea0fed1e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.085637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59b2ac-77ec-4c42-9cf1-dea0fed1e09c admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.085918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59b2ac-77ec-4c42-9cf1-dea0fed1e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.086192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59b2ac-77ec-4c42-9cf1-dea0fed1e09c admin admin] PATH_INFO: `/v3/projects/a3f4bf2cb16e453489564f4ef1c5c166/users/d52d330588164be2b58e8f042aa6bd27/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.100897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59b2ac-77ec-4c42-9cf1-dea0fed1e09c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f4bf2cb16e453489564f4ef1c5c166, user_id=d52d330588164be2b58e8f042aa6bd27, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.103550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59b2ac-77ec-4c42-9cf1-dea0fed1e09c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.170632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 902/1797] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:31 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:59:32.171827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397fe3b-30f0-408a-8f77-8251b2f2b62d admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.189367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 896/1798] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:32 2022] PUT /identity/v3/projects/a3f4bf2cb16e453489564f4ef1c5c166/users/d52d330588164be2b58e8f042aa6bd27/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:32.199572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731871-451c-41c6-b91d-b8000a9a4427 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.199861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731871-451c-41c6-b91d-b8000a9a4427 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.200144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731871-451c-41c6-b91d-b8000a9a4427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.227467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a5731871-451c-41c6-b91d-b8000a9a4427 None None] MFA Rules not processed for user `d52d330588164be2b58e8f042aa6bd27`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:32.228843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a5731871-451c-41c6-b91d-b8000a9a4427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:32.243371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397fe3b-30f0-408a-8f77-8251b2f2b62d admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.267523 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:32.329012 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 897/1799] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:32.329116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c329c23-00de-43b7-9122-7f38666a90c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.329116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c329c23-00de-43b7-9122-7f38666a90c0 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.329116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c329c23-00de-43b7-9122-7f38666a90c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.329116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c329c23-00de-43b7-9122-7f38666a90c0 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.329454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c329c23-00de-43b7-9122-7f38666a90c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.330394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5731871-451c-41c6-b91d-b8000a9a4427 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.330705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c329c23-00de-43b7-9122-7f38666a90c0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.346034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5731871-451c-41c6-b91d-b8000a9a4427 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.352621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f845c3-35e0-47c1-bf2f-5fb5bc815fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.353480 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f845c3-35e0-47c1-bf2f-5fb5bc815fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.353763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f845c3-35e0-47c1-bf2f-5fb5bc815fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.354040 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f845c3-35e0-47c1-bf2f-5fb5bc815fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.354764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f845c3-35e0-47c1-bf2f-5fb5bc815fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.357856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f845c3-35e0-47c1-bf2f-5fb5bc815fa8 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.424778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 903/1800] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:32 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:32.428467 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c329c23-00de-43b7-9122-7f38666a90c0 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.444778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c329c23-00de-43b7-9122-7f38666a90c0 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.448516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 898/1801] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:32 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:32.451380 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508f8c2-0ee6-4234-9e0a-882cd23207f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.452414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508f8c2-0ee6-4234-9e0a-882cd23207f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.452582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508f8c2-0ee6-4234-9e0a-882cd23207f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.452838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f845c3-35e0-47c1-bf2f-5fb5bc815fa8 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.452958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508f8c2-0ee6-4234-9e0a-882cd23207f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.453687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508f8c2-0ee6-4234-9e0a-882cd23207f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.456438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508f8c2-0ee6-4234-9e0a-882cd23207f9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.468854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 904/1802] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:32.472998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508f8c2-0ee6-4234-9e0a-882cd23207f9 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.484534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508f8c2-0ee6-4234-9e0a-882cd23207f9 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.523399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:32.540038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f845c3-35e0-47c1-bf2f-5fb5bc815fa8 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.555511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf03726-6940-4256-8b95-4a3dba4fcf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.557306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf03726-6940-4256-8b95-4a3dba4fcf14 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.557790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf03726-6940-4256-8b95-4a3dba4fcf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.558269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf03726-6940-4256-8b95-4a3dba4fcf14 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.559307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf03726-6940-4256-8b95-4a3dba4fcf14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.562078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf03726-6940-4256-8b95-4a3dba4fcf14 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.573659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7507-d40d-4516-b647-d47e59114b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.584242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7507-d40d-4516-b647-d47e59114b67 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.585219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7507-d40d-4516-b647-d47e59114b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.585219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7507-d40d-4516-b647-d47e59114b67 admin admin] PATH_INFO: `/v3/projects/397a7f96f3fc4680926ba227af78cb4d/users/bc411f39049348d181c6f6cc0757b3fb/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.605883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f7507-d40d-4516-b647-d47e59114b67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=397a7f96f3fc4680926ba227af78cb4d, user_id=bc411f39049348d181c6f6cc0757b3fb, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.610018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f7507-d40d-4516-b647-d47e59114b67 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.658738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 899/1803] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:32 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:32.663569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf03726-6940-4256-8b95-4a3dba4fcf14 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.687147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf03726-6940-4256-8b95-4a3dba4fcf14 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.698267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 905/1804] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:32 2022] PUT /identity/v3/projects/397a7f96f3fc4680926ba227af78cb4d/users/bc411f39049348d181c6f6cc0757b3fb/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:32.702507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7507-d40d-4516-b647-d47e59114b67 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.761938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:32.794659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779eea4-29a4-4288-8a6c-8f5a407460e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.795487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779eea4-29a4-4288-8a6c-8f5a407460e2 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.795928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779eea4-29a4-4288-8a6c-8f5a407460e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.797278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779eea4-29a4-4288-8a6c-8f5a407460e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.798486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779eea4-29a4-4288-8a6c-8f5a407460e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.801117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779eea4-29a4-4288-8a6c-8f5a407460e2 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.818520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 900/1805] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:32.821426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779eea4-29a4-4288-8a6c-8f5a407460e2 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.825078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7507-d40d-4516-b647-d47e59114b67 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.831507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4035896-cc3f-47dd-b902-e7800c6f0f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.832436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4035896-cc3f-47dd-b902-e7800c6f0f35 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.833983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4035896-cc3f-47dd-b902-e7800c6f0f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.834438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4035896-cc3f-47dd-b902-e7800c6f0f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.835494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4035896-cc3f-47dd-b902-e7800c6f0f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.838695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4035896-cc3f-47dd-b902-e7800c6f0f35 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.849023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779eea4-29a4-4288-8a6c-8f5a407460e2 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.857581 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 906/1806] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:32.858648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4035896-cc3f-47dd-b902-e7800c6f0f35 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:32.871641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4035896-cc3f-47dd-b902-e7800c6f0f35 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:32.873250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf349d-9f53-4ab6-bf4c-122395592923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.874240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf349d-9f53-4ab6-bf4c-122395592923 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.875201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf349d-9f53-4ab6-bf4c-122395592923 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.875201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf349d-9f53-4ab6-bf4c-122395592923 admin admin] PATH_INFO: `/v3/projects/7f8cc90e92644912b5e74e8f11747244/users/b51b95f8a1dc49d99b7efcd814e02e19/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.877602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc7d83-af6c-4a62-b6a8-c209285b1b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:32.878728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc7d83-af6c-4a62-b6a8-c209285b1b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.879015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc7d83-af6c-4a62-b6a8-c209285b1b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.879295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc7d83-af6c-4a62-b6a8-c209285b1b87 admin admin] PATH_INFO: `/v3/projects/397a7f96f3fc4680926ba227af78cb4d/users/bc411f39049348d181c6f6cc0757b3fb/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.882882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc7d83-af6c-4a62-b6a8-c209285b1b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=397a7f96f3fc4680926ba227af78cb4d, user_id=bc411f39049348d181c6f6cc0757b3fb, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.885638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc7d83-af6c-4a62-b6a8-c209285b1b87 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.899821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 907/1807] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:32 2022] PUT /identity/v3/projects/397a7f96f3fc4680926ba227af78cb4d/users/bc411f39049348d181c6f6cc0757b3fb/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:32.911830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc192783-1e83-4764-b9ba-5316f905988d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:32.912412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc192783-1e83-4764-b9ba-5316f905988d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:32.913257 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc192783-1e83-4764-b9ba-5316f905988d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:32.914908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf349d-9f53-4ab6-bf4c-122395592923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f8cc90e92644912b5e74e8f11747244, user_id=b51b95f8a1dc49d99b7efcd814e02e19, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:32.918869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf349d-9f53-4ab6-bf4c-122395592923 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:32.942185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-bc192783-1e83-4764-b9ba-5316f905988d None None] MFA Rules not processed for user `bc411f39049348d181c6f6cc0757b3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:32.943514 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bc192783-1e83-4764-b9ba-5316f905988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:32.996083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 901/1808] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:32 2022] PUT /identity/v3/projects/7f8cc90e92644912b5e74e8f11747244/users/b51b95f8a1dc49d99b7efcd814e02e19/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:32.997097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf349d-9f53-4ab6-bf4c-122395592923 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:33.069457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 908/1809] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:33.083972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc192783-1e83-4764-b9ba-5316f905988d None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:33.085599 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bc192783-1e83-4764-b9ba-5316f905988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:33.088360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf349d-9f53-4ab6-bf4c-122395592923 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:33.097965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a430946-c68e-4df6-bd7d-8471b99c234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:33.097965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a430946-c68e-4df6-bd7d-8471b99c234e admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:33.097965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a430946-c68e-4df6-bd7d-8471b99c234e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:33.097965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a430946-c68e-4df6-bd7d-8471b99c234e admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:33.098370 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a430946-c68e-4df6-bd7d-8471b99c234e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:33.100855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a430946-c68e-4df6-bd7d-8471b99c234e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:33.102915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:33.112075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 902/1810] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:33.116047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a430946-c68e-4df6-bd7d-8471b99c234e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:33.126077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a430946-c68e-4df6-bd7d-8471b99c234e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:33.131801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922cd42-db46-4464-9b3a-970be6ba4067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:33.132878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922cd42-db46-4464-9b3a-970be6ba4067 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:33.133225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922cd42-db46-4464-9b3a-970be6ba4067 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:33.133569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922cd42-db46-4464-9b3a-970be6ba4067 admin admin] PATH_INFO: `/v3/projects/7f8cc90e92644912b5e74e8f11747244/users/b51b95f8a1dc49d99b7efcd814e02e19/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:33.135743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922cd42-db46-4464-9b3a-970be6ba4067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f8cc90e92644912b5e74e8f11747244, user_id=b51b95f8a1dc49d99b7efcd814e02e19, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:33.138856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922cd42-db46-4464-9b3a-970be6ba4067 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:33.152602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 903/1811] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:33 2022] PUT /identity/v3/projects/7f8cc90e92644912b5e74e8f11747244/users/b51b95f8a1dc49d99b7efcd814e02e19/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:33.164242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc192783-1e83-4764-b9ba-5316f905988d None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:33.166272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3ad77-b334-4d96-9d94-2bfefda75ebb None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:33.166713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3ad77-b334-4d96-9d94-2bfefda75ebb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:33.166992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3ad77-b334-4d96-9d94-2bfefda75ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:33.196737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f5b3ad77-b334-4d96-9d94-2bfefda75ebb None None] MFA Rules not processed for user `b51b95f8a1dc49d99b7efcd814e02e19`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:33.198007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f5b3ad77-b334-4d96-9d94-2bfefda75ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:33.223184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bfe05-8fe9-49db-a61b-f2f352aad475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:33.223840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bfe05-8fe9-49db-a61b-f2f352aad475 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:33.226457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bfe05-8fe9-49db-a61b-f2f352aad475 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:33.226457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bfe05-8fe9-49db-a61b-f2f352aad475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:33.270192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 904/1812] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:33.282179 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3ad77-b334-4d96-9d94-2bfefda75ebb None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:33.283585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f5b3ad77-b334-4d96-9d94-2bfefda75ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:33.285171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bfe05-8fe9-49db-a61b-f2f352aad475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:33.286635 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bfe05-8fe9-49db-a61b-f2f352aad475 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:33.296331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3ad77-b334-4d96-9d94-2bfefda75ebb None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:33.300180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 909/1813] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:33.303221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2ed63-9761-4bea-a91f-bab7b35f9744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:33.303889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2ed63-9761-4bea-a91f-bab7b35f9744 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:33.304588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2ed63-9761-4bea-a91f-bab7b35f9744 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:33.304588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2ed63-9761-4bea-a91f-bab7b35f9744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:33.315213 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2ed63-9761-4bea-a91f-bab7b35f9744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:33.316871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2ed63-9761-4bea-a91f-bab7b35f9744 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:33.326731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 905/1814] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:33.356838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bfe05-8fe9-49db-a61b-f2f352aad475 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:33.424842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bfe05-8fe9-49db-a61b-f2f352aad475 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:33.429689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:33.430644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:33.430929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:33.431255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] PATH_INFO: `/v3/users/d52d330588164be2b58e8f042aa6bd27` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:33.432455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52d330588164be2b58e8f042aa6bd27)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:33.434540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:33.457488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:33.674199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52d330588164be2b58e8f042aa6bd27'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:33.681205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52d330588164be2b58e8f042aa6bd27'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:33.681872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52d330588164be2b58e8f042aa6bd27'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:33.794509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52d330588164be2b58e8f042aa6bd27'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:33.796296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52d330588164be2b58e8f042aa6bd27'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:33.817754 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 910/1815] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:33 2022] DELETE /identity/v3/users/d52d330588164be2b58e8f042aa6bd27 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:33.822403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93ef7b-3dce-4de8-bd00-d1f0fca9c323 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:33.822915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93ef7b-3dce-4de8-bd00-d1f0fca9c323 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:33.823031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93ef7b-3dce-4de8-bd00-d1f0fca9c323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:33.831405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:33.849713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-ab93ef7b-3dce-4de8-bd00-d1f0fca9c323 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:33.850949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ab93ef7b-3dce-4de8-bd00-d1f0fca9c323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:33.923453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 906/1816] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:33.933026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:33.935040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:33.999420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dcfd5-c8b8-41d1-8ff1-7933264227ea admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:34.005717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299764a6-f821-472e-bd7b-6d1e8359224b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:34.006918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299764a6-f821-472e-bd7b-6d1e8359224b service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:34.007471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299764a6-f821-472e-bd7b-6d1e8359224b service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:34.007840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299764a6-f821-472e-bd7b-6d1e8359224b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:34.018597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299764a6-f821-472e-bd7b-6d1e8359224b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:34.026075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299764a6-f821-472e-bd7b-6d1e8359224b service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:34.033393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 911/1817] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:34.416277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93ef7b-3dce-4de8-bd00-d1f0fca9c323 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:34.418177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ab93ef7b-3dce-4de8-bd00-d1f0fca9c323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:34.552056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93ef7b-3dce-4de8-bd00-d1f0fca9c323 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:34.571997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c35610-c1f9-448d-8713-ffb790607304 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc411f39049348d181c6f6cc0757b3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '397a7f96f3fc4680926ba227af78cb4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:34.572893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35610-c1f9-448d-8713-ffb790607304 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:34.573330 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35610-c1f9-448d-8713-ffb790607304 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:34.573891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35610-c1f9-448d-8713-ffb790607304 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:34.578508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c35610-c1f9-448d-8713-ffb790607304 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:34.579043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c35610-c1f9-448d-8713-ffb790607304 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:34.581188 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c35610-c1f9-448d-8713-ffb790607304 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:34.605990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 907/1818] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:34.611217 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299764a6-f821-472e-bd7b-6d1e8359224b service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:34.622961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299764a6-f821-472e-bd7b-6d1e8359224b service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:34.627958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a1ab2-8f63-45aa-af7b-ea4591c7f498 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc411f39049348d181c6f6cc0757b3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '397a7f96f3fc4680926ba227af78cb4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:34.636840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a1ab2-8f63-45aa-af7b-ea4591c7f498 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:34.636840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a1ab2-8f63-45aa-af7b-ea4591c7f498 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:34.636840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a1ab2-8f63-45aa-af7b-ea4591c7f498 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:34.636840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a1ab2-8f63-45aa-af7b-ea4591c7f498 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:34.636840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a1ab2-8f63-45aa-af7b-ea4591c7f498 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:34.707145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c35610-c1f9-448d-8713-ffb790607304 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:34.708927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e2c35610-c1f9-448d-8713-ffb790607304 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:34.725768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 912/1819] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:34 2022] POST /identity/v3/projects => generated 354 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:34.732200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a1ab2-8f63-45aa-af7b-ea4591c7f498 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:34.733675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c35610-c1f9-448d-8713-ffb790607304 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:34.736261 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0e7a1ab2-8f63-45aa-af7b-ea4591c7f498 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:34.803821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:34.804730 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:34.805015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:34.805320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] PATH_INFO: `/v3/users/677031e79ac04fe385988f107ef67a77` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:34.806693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=677031e79ac04fe385988f107ef67a77)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:34.808659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:34.833146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a1ab2-8f63-45aa-af7b-ea4591c7f498 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:34.838581 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:34.839511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:34.839790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:34.840065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] PATH_INFO: `/v3/projects/a3f4bf2cb16e453489564f4ef1c5c166` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:34.841366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f4bf2cb16e453489564f4ef1c5c166)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:34.843864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:34.872322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:35.047286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:35.060853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f4bf2cb16e453489564f4ef1c5c166', 'request_id': 'req-7620f5ef-d4c8-4605-8a3e-10fd635c5852'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.063137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677031e79ac04fe385988f107ef67a77'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.081189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f4bf2cb16e453489564f4ef1c5c166', 'request_id': 'req-7620f5ef-d4c8-4605-8a3e-10fd635c5852'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.088216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677031e79ac04fe385988f107ef67a77'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.092083 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f4bf2cb16e453489564f4ef1c5c166', 'request_id': 'req-7620f5ef-d4c8-4605-8a3e-10fd635c5852'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.166029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677031e79ac04fe385988f107ef67a77'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.166829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677031e79ac04fe385988f107ef67a77'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.167464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677031e79ac04fe385988f107ef67a77'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.186113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 908/1820] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:34 2022] DELETE /identity/v3/users/677031e79ac04fe385988f107ef67a77 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:35.189122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:35.190066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:35.216899 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] The token cache is being invalidate because project a3f4bf2cb16e453489564f4ef1c5c166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:35.217253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.217892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7620f5ef-d4c8-4605-8a3e-10fd635c5852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.220905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 913/1821] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:34 2022] DELETE /identity/v3/projects/a3f4bf2cb16e453489564f4ef1c5c166 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:35.223055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157333a8-14ae-4194-a8f6-5561ae962ace None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:35.224122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157333a8-14ae-4194-a8f6-5561ae962ace None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:35.224488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157333a8-14ae-4194-a8f6-5561ae962ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:35.253538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-157333a8-14ae-4194-a8f6-5561ae962ace None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:35.255128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-157333a8-14ae-4194-a8f6-5561ae962ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:35.282881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09fca6-7c02-4976-bdf6-74a459d54314 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:35.288770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa55e1-c46f-4484-bf45-7c8e9c7cbc9f tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc411f39049348d181c6f6cc0757b3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '397a7f96f3fc4680926ba227af78cb4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:35.289300 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa55e1-c46f-4484-bf45-7c8e9c7cbc9f tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:35.297231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa55e1-c46f-4484-bf45-7c8e9c7cbc9f tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:35.297231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa55e1-c46f-4484-bf45-7c8e9c7cbc9f tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:35.297231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa55e1-c46f-4484-bf45-7c8e9c7cbc9f tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:35.297231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa55e1-c46f-4484-bf45-7c8e9c7cbc9f tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:35.297231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa55e1-c46f-4484-bf45-7c8e9c7cbc9f tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:35.308537 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 909/1822] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:35.309296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa55e1-c46f-4484-bf45-7c8e9c7cbc9f tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:35.311543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4efa55e1-c46f-4484-bf45-7c8e9c7cbc9f tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:35.371522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 914/1823] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:35.372415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157333a8-14ae-4194-a8f6-5561ae962ace None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:35.374519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-157333a8-14ae-4194-a8f6-5561ae962ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:35.394369 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157333a8-14ae-4194-a8f6-5561ae962ace None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:35.399601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc411f39049348d181c6f6cc0757b3fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '397a7f96f3fc4680926ba227af78cb4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:35.400563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:35.400855 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:35.401133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] PATH_INFO: `/v3/projects/02258c80efaa4d5dbeea60dfc6c48993` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:35.402344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: Authorizing `identity:delete_project(project_id=02258c80efaa4d5dbeea60dfc6c48993)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:35.402923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa55e1-c46f-4484-bf45-7c8e9c7cbc9f tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:35.404294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:35.409047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:35.410457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:35.411342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:35.411342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] PATH_INFO: `/v3/users/b51b95f8a1dc49d99b7efcd814e02e19` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:35.412745 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b51b95f8a1dc49d99b7efcd814e02e19)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:35.414877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:35.624593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b51b95f8a1dc49d99b7efcd814e02e19'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.629584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02258c80efaa4d5dbeea60dfc6c48993', 'request_id': 'req-649edcb1-d7ea-418e-8f80-0065a0f63a60'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.631473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:35.634160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b51b95f8a1dc49d99b7efcd814e02e19'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.637508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02258c80efaa4d5dbeea60dfc6c48993', 'request_id': 'req-649edcb1-d7ea-418e-8f80-0065a0f63a60'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.645397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02258c80efaa4d5dbeea60dfc6c48993', 'request_id': 'req-649edcb1-d7ea-418e-8f80-0065a0f63a60'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.715460 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b51b95f8a1dc49d99b7efcd814e02e19'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.715653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b51b95f8a1dc49d99b7efcd814e02e19'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.716464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b51b95f8a1dc49d99b7efcd814e02e19'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.739531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 910/1824] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:35 2022] DELETE /identity/v3/users/b51b95f8a1dc49d99b7efcd814e02e19 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:35.740865 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:35.742671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:35.812476 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] The token cache is being invalidate because project 02258c80efaa4d5dbeea60dfc6c48993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:35.813062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.813986 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:35.816557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 915/1825] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:35 2022] DELETE /identity/v3/projects/02258c80efaa4d5dbeea60dfc6c48993 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:35.818343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3d1d-48b0-425e-952e-dd51eea6aa13 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:35.825707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153617d8-d1a0-4b88-91eb-10be22421e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:35.826595 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153617d8-d1a0-4b88-91eb-10be22421e00 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:35.826866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153617d8-d1a0-4b88-91eb-10be22421e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:35.827183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153617d8-d1a0-4b88-91eb-10be22421e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:35.830165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-153617d8-d1a0-4b88-91eb-10be22421e00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:35.913694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153617d8-d1a0-4b88-91eb-10be22421e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:35.915176 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153617d8-d1a0-4b88-91eb-10be22421e00 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:35.924795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 911/1826] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:36.334722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:36.336880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:36.379580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:36.430218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649edcb1-d7ea-418e-8f80-0065a0f63a60 tempest-VolumeQuotaClassesTest-1298552321 tempest-VolumeQuotaClassesTest-1298552321-project-admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:36.444299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:36.444299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:36.444299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:36.444299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] PATH_INFO: `/v3/projects/7f8cc90e92644912b5e74e8f11747244` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:36.444595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f8cc90e92644912b5e74e8f11747244)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:36.445652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153617d8-d1a0-4b88-91eb-10be22421e00 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:36.447137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-153617d8-d1a0-4b88-91eb-10be22421e00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:36.447394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:36.476590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153617d8-d1a0-4b88-91eb-10be22421e00 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:36.480465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:36.481495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:36.481855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:36.482233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] PATH_INFO: `/v3/projects/06767cd5344d4d4bac5d4903001d0e28` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:36.483688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06767cd5344d4d4bac5d4903001d0e28)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:36.486289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:36.570210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f8cc90e92644912b5e74e8f11747244', 'request_id': 'req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:36.577329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f8cc90e92644912b5e74e8f11747244', 'request_id': 'req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:36.583979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f8cc90e92644912b5e74e8f11747244', 'request_id': 'req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:36.636777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06767cd5344d4d4bac5d4903001d0e28', 'request_id': 'req-d0dcdb5a-128f-4ff6-b028-d3c756678a21'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:36.645172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06767cd5344d4d4bac5d4903001d0e28', 'request_id': 'req-d0dcdb5a-128f-4ff6-b028-d3c756678a21'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:36.651677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06767cd5344d4d4bac5d4903001d0e28', 'request_id': 'req-d0dcdb5a-128f-4ff6-b028-d3c756678a21'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:36.676034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] The token cache is being invalidate because project 7f8cc90e92644912b5e74e8f11747244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:36.676332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:36.676980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:36.678610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 916/1827] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:36 2022] DELETE /identity/v3/projects/7f8cc90e92644912b5e74e8f11747244 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:36.679395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:36.680778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:36.747767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938b24e-6d07-4a2c-b67f-9c1e9ae82c4f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:36.755331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] The token cache is being invalidate because project 06767cd5344d4d4bac5d4903001d0e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:36.755634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:36.756278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d0dcdb5a-128f-4ff6-b028-d3c756678a21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:36.759962 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 912/1828] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:36 2022] DELETE /identity/v3/projects/06767cd5344d4d4bac5d4903001d0e28 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:36.761027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e20b5-5dbe-4471-8197-b94f14eaf555 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:36.761319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e20b5-5dbe-4471-8197-b94f14eaf555 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:36.761601 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e20b5-5dbe-4471-8197-b94f14eaf555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:36.772066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:36.783999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-ce7e20b5-5dbe-4471-8197-b94f14eaf555 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:36.785221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ce7e20b5-5dbe-4471-8197-b94f14eaf555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:36.832250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:36.833223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:36.833505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:36.833784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] PATH_INFO: `/v3/users/ee63049c8c7b42a8a77e3728d768e5dc` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:36.835072 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee63049c8c7b42a8a77e3728d768e5dc)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:36.837075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:36.849150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 913/1829] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:36.856410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e20b5-5dbe-4471-8197-b94f14eaf555 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:36.857732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ce7e20b5-5dbe-4471-8197-b94f14eaf555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:36.872633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e20b5-5dbe-4471-8197-b94f14eaf555 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:36.878024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:36.879141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:36.880791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:36.880791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] PATH_INFO: `/v3/users/92d119d820ba4182a1eee4da178a07eb` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:36.882297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92d119d820ba4182a1eee4da178a07eb)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:36.884238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:37.032939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee63049c8c7b42a8a77e3728d768e5dc'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:37.051624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee63049c8c7b42a8a77e3728d768e5dc'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:37.051624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee63049c8c7b42a8a77e3728d768e5dc'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:37.108625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee63049c8c7b42a8a77e3728d768e5dc'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:37.109625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1c7d7d7d-a1ca-4667-9e99-6571b4560120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee63049c8c7b42a8a77e3728d768e5dc'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:37.132955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92d119d820ba4182a1eee4da178a07eb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:37.140813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92d119d820ba4182a1eee4da178a07eb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:37.149479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 917/1830] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:36 2022] DELETE /identity/v3/users/ee63049c8c7b42a8a77e3728d768e5dc => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:37.152026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb63d7-5b23-46b2-81de-69e84376017c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.152445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb63d7-5b23-46b2-81de-69e84376017c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.152762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb63d7-5b23-46b2-81de-69e84376017c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.171235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-99bb63d7-5b23-46b2-81de-69e84376017c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:37.172502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-99bb63d7-5b23-46b2-81de-69e84376017c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:37.211994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92d119d820ba4182a1eee4da178a07eb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:37.212677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92d119d820ba4182a1eee4da178a07eb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:37.214089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-17a50d83-04b4-4684-9d9f-3b3a82aa73e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92d119d820ba4182a1eee4da178a07eb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:37.249245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 914/1831] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:36 2022] DELETE /identity/v3/users/92d119d820ba4182a1eee4da178a07eb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:37.252708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c339e0-8c7d-46b6-a7db-7ab2a211c3f9 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.252708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c339e0-8c7d-46b6-a7db-7ab2a211c3f9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.252708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c339e0-8c7d-46b6-a7db-7ab2a211c3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.259491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 918/1832] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:37.263449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb63d7-5b23-46b2-81de-69e84376017c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:37.284823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb63d7-5b23-46b2-81de-69e84376017c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:37.295972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:37.330655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f8c339e0-8c7d-46b6-a7db-7ab2a211c3f9 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:37.332035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f8c339e0-8c7d-46b6-a7db-7ab2a211c3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:37.402256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d02ab6-7086-40c1-8749-0821b43ec971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:37.402971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d02ab6-7086-40c1-8749-0821b43ec971 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.403249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d02ab6-7086-40c1-8749-0821b43ec971 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.403525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d02ab6-7086-40c1-8749-0821b43ec971 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.404405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d02ab6-7086-40c1-8749-0821b43ec971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:37.404743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d02ab6-7086-40c1-8749-0821b43ec971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:37.406359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d02ab6-7086-40c1-8749-0821b43ec971 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:37.424458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 919/1833] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:37.425382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d02ab6-7086-40c1-8749-0821b43ec971 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:37.435727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d02ab6-7086-40c1-8749-0821b43ec971 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:37.441373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d3c3-dd1e-49f6-aec0-e3876ae36ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:37.442039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d3c3-dd1e-49f6-aec0-e3876ae36ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.442321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d3c3-dd1e-49f6-aec0-e3876ae36ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.442665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776d3c3-dd1e-49f6-aec0-e3876ae36ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.444727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776d3c3-dd1e-49f6-aec0-e3876ae36ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:37.445372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776d3c3-dd1e-49f6-aec0-e3876ae36ea9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:37.458580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 915/1834] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:37.472553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c339e0-8c7d-46b6-a7db-7ab2a211c3f9 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:37.474343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f8c339e0-8c7d-46b6-a7db-7ab2a211c3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:37.539915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c339e0-8c7d-46b6-a7db-7ab2a211c3f9 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:37.543976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273aafb-31f5-4b64-b222-8dd996140a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:37.544757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273aafb-31f5-4b64-b222-8dd996140a8f service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.548774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273aafb-31f5-4b64-b222-8dd996140a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.548774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273aafb-31f5-4b64-b222-8dd996140a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.557086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273aafb-31f5-4b64-b222-8dd996140a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:37.558893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273aafb-31f5-4b64-b222-8dd996140a8f service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:37.567152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:37.573298 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 916/1835] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:37.578632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 920/1836] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:37 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:37.582454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273aafb-31f5-4b64-b222-8dd996140a8f service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:37.599437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273aafb-31f5-4b64-b222-8dd996140a8f service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:37.673599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5fe67-959f-4a5f-8114-6c68865b24ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:37.674902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5fe67-959f-4a5f-8114-6c68865b24ee admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.674902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5fe67-959f-4a5f-8114-6c68865b24ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.675080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5fe67-959f-4a5f-8114-6c68865b24ee admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.675702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5fe67-959f-4a5f-8114-6c68865b24ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:37.677752 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5fe67-959f-4a5f-8114-6c68865b24ee admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:37.847323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 917/1837] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:37 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:37.851768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d3c3-dd1e-49f6-aec0-e3876ae36ea9 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:37.880380 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776d3c3-dd1e-49f6-aec0-e3876ae36ea9 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:37.887696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35257a3f-4007-431f-97cd-4c6f0a399b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:37.888666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35257a3f-4007-431f-97cd-4c6f0a399b22 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.889024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35257a3f-4007-431f-97cd-4c6f0a399b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.890559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35257a3f-4007-431f-97cd-4c6f0a399b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.894522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35257a3f-4007-431f-97cd-4c6f0a399b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:37.894522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35257a3f-4007-431f-97cd-4c6f0a399b22 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:37.905917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 921/1838] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:37.909996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5fe67-959f-4a5f-8114-6c68865b24ee admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:37.920391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5fe67-959f-4a5f-8114-6c68865b24ee admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:37.930897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115712c3-dbb3-4d7e-b941-d8cac65392d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:37.930897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115712c3-dbb3-4d7e-b941-d8cac65392d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.930897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115712c3-dbb3-4d7e-b941-d8cac65392d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.930897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115712c3-dbb3-4d7e-b941-d8cac65392d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.931327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115712c3-dbb3-4d7e-b941-d8cac65392d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:37.931327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115712c3-dbb3-4d7e-b941-d8cac65392d3 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:37.941635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 918/1839] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:37.945859 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35257a3f-4007-431f-97cd-4c6f0a399b22 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:37.959436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35257a3f-4007-431f-97cd-4c6f0a399b22 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:37.960921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9948bd9-59ab-4cf7-a8f9-641bf0591b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:37.961846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9948bd9-59ab-4cf7-a8f9-641bf0591b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.962126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9948bd9-59ab-4cf7-a8f9-641bf0591b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.962398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9948bd9-59ab-4cf7-a8f9-641bf0591b58 admin admin] PATH_INFO: `/v3/projects/9654aefbc1874980b8b5a7dc663817d8/users/32b9351e6e5049898295654bcd824ef6/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.979826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9948bd9-59ab-4cf7-a8f9-641bf0591b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9654aefbc1874980b8b5a7dc663817d8, user_id=32b9351e6e5049898295654bcd824ef6, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:37.989928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9948bd9-59ab-4cf7-a8f9-641bf0591b58 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:38.053532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 922/1840] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:37 2022] PUT /identity/v3/projects/9654aefbc1874980b8b5a7dc663817d8/users/32b9351e6e5049898295654bcd824ef6/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:38.064339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbe999-3b2d-4e23-965c-7c04e71c3ce0 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.066143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbe999-3b2d-4e23-965c-7c04e71c3ce0 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.066603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbe999-3b2d-4e23-965c-7c04e71c3ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:38.073182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9948bd9-59ab-4cf7-a8f9-641bf0591b58 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:38.075015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e9948bd9-59ab-4cf7-a8f9-641bf0591b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:38.117004 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-b2bbe999-3b2d-4e23-965c-7c04e71c3ce0 None None] MFA Rules not processed for user `32b9351e6e5049898295654bcd824ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:38.121503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b2bbe999-3b2d-4e23-965c-7c04e71c3ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:38.210689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9948bd9-59ab-4cf7-a8f9-641bf0591b58 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:38.215984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:38.216861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 919/1841] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:38.222677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.223234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.223665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] PATH_INFO: `/v3/projects/c2f15693b89b47c4a72fe5485d5afb65` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:38.225727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f15693b89b47c4a72fe5485d5afb65)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:38.228293 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:38.246645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbe999-3b2d-4e23-965c-7c04e71c3ce0 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:38.248324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b2bbe999-3b2d-4e23-965c-7c04e71c3ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:38.314284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbe999-3b2d-4e23-965c-7c04e71c3ce0 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:38.317935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8745bf1-88c3-4ef7-90e1-55929c479f04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:38.318967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8745bf1-88c3-4ef7-90e1-55929c479f04 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.319533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8745bf1-88c3-4ef7-90e1-55929c479f04 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.320103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8745bf1-88c3-4ef7-90e1-55929c479f04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:38.335699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8745bf1-88c3-4ef7-90e1-55929c479f04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:38.337154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8745bf1-88c3-4ef7-90e1-55929c479f04 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:38.349404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 920/1842] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:38.367413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f15693b89b47c4a72fe5485d5afb65', 'request_id': 'req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.375630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8745bf1-88c3-4ef7-90e1-55929c479f04 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:38.378697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f15693b89b47c4a72fe5485d5afb65', 'request_id': 'req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.385420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f15693b89b47c4a72fe5485d5afb65', 'request_id': 'req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.386779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8745bf1-88c3-4ef7-90e1-55929c479f04 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:38.444663 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:38.445548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.445831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.446107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] PATH_INFO: `/v3/users/32b9351e6e5049898295654bcd824ef6` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:38.447363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b9351e6e5049898295654bcd824ef6)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:38.449300 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:38.480197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] The token cache is being invalidate because project c2f15693b89b47c4a72fe5485d5afb65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:38.480715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.481605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.483247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 923/1843] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:38 2022] DELETE /identity/v3/projects/c2f15693b89b47c4a72fe5485d5afb65 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:38.487440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:38.488843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:38.506842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a3dee-5607-43b9-8465-8bf7ca09e89a admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:38.524817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:38.524817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.524817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.524817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] PATH_INFO: `/v3/users/bc411f39049348d181c6f6cc0757b3fb` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:38.525915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc411f39049348d181c6f6cc0757b3fb)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:38.525915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:38.652533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b9351e6e5049898295654bcd824ef6'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.660825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b9351e6e5049898295654bcd824ef6'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.755046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b9351e6e5049898295654bcd824ef6'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.755783 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b9351e6e5049898295654bcd824ef6'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.756392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-36be109a-d4ab-4688-9361-d6d39ca941cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b9351e6e5049898295654bcd824ef6'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.772195 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc411f39049348d181c6f6cc0757b3fb'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.780985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc411f39049348d181c6f6cc0757b3fb'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.782083 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc411f39049348d181c6f6cc0757b3fb'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.782204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 921/1844] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:38 2022] DELETE /identity/v3/users/32b9351e6e5049898295654bcd824ef6 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:38.785983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:38.789897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2d6d4-ff63-43f4-8b56-865e4ce93e1d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.790201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2d6d4-ff63-43f4-8b56-865e4ce93e1d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.792434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2d6d4-ff63-43f4-8b56-865e4ce93e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:38.816479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-61d2d6d4-ff63-43f4-8b56-865e4ce93e1d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:38.817887 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-61d2d6d4-ff63-43f4-8b56-865e4ce93e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:38.868709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc411f39049348d181c6f6cc0757b3fb'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.869358 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc411f39049348d181c6f6cc0757b3fb'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.897299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 922/1845] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:38.900297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 924/1846] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:38 2022] DELETE /identity/v3/users/bc411f39049348d181c6f6cc0757b3fb => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:38.921587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2d6d4-ff63-43f4-8b56-865e4ce93e1d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:38.923536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-61d2d6d4-ff63-43f4-8b56-865e4ce93e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:38.963182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:38.964648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:39.005276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2d6d4-ff63-43f4-8b56-865e4ce93e1d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:39.098658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696aa43e-fb5e-416e-b5eb-dfaec510838e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:39.112866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:39.115426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.115426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.115426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] PATH_INFO: `/v3/projects/70173cb4a9a645e7985642fde941450d` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.116277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fca6f-8266-48d7-a1a4-dd62a68fa414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:39.117255 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fca6f-8266-48d7-a1a4-dd62a68fa414 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.117677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] RBAC: Authorizing `identity:delete_project(project_id=70173cb4a9a645e7985642fde941450d)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:39.117853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fca6f-8266-48d7-a1a4-dd62a68fa414 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.118273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fca6f-8266-48d7-a1a4-dd62a68fa414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.119668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:39.132520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087fca6f-8266-48d7-a1a4-dd62a68fa414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:39.134281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087fca6f-8266-48d7-a1a4-dd62a68fa414 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:39.150105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 923/1847] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:39.161225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:39.234736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70173cb4a9a645e7985642fde941450d', 'request_id': 'req-705cf6da-668f-4657-9fc6-75b1868e815f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:39.243579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70173cb4a9a645e7985642fde941450d', 'request_id': 'req-705cf6da-668f-4657-9fc6-75b1868e815f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:39.249585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70173cb4a9a645e7985642fde941450d', 'request_id': 'req-705cf6da-668f-4657-9fc6-75b1868e815f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:39.328424 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] The token cache is being invalidate because project 70173cb4a9a645e7985642fde941450d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:39.329027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:39.329869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-705cf6da-668f-4657-9fc6-75b1868e815f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:39.331984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 925/1848] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:38 2022] DELETE /identity/v3/projects/70173cb4a9a645e7985642fde941450d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:39.336836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0405c4-8c3b-41d9-bada-97a559d4cf5d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.337160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0405c4-8c3b-41d9-bada-97a559d4cf5d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.337403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0405c4-8c3b-41d9-bada-97a559d4cf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.358139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-2c0405c4-8c3b-41d9-bada-97a559d4cf5d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:39.359611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2c0405c4-8c3b-41d9-bada-97a559d4cf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:39.428874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 924/1849] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:39.437544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cdab9-521a-42a1-a110-73e92ad5ffad None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.438147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cdab9-521a-42a1-a110-73e92ad5ffad None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.438513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cdab9-521a-42a1-a110-73e92ad5ffad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.460029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e10cdab9-521a-42a1-a110-73e92ad5ffad None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:39.461338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e10cdab9-521a-42a1-a110-73e92ad5ffad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:39.470433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 926/1850] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:39.475248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0405c4-8c3b-41d9-bada-97a559d4cf5d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:39.489028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0405c4-8c3b-41d9-bada-97a559d4cf5d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:39.494049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d58dde-a867-4f84-88e9-100f71d27e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:39.494803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d58dde-a867-4f84-88e9-100f71d27e70 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.495086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d58dde-a867-4f84-88e9-100f71d27e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.495376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d58dde-a867-4f84-88e9-100f71d27e70 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.496140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d58dde-a867-4f84-88e9-100f71d27e70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:39.496464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d58dde-a867-4f84-88e9-100f71d27e70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:39.498176 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d58dde-a867-4f84-88e9-100f71d27e70 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:39.525950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cdab9-521a-42a1-a110-73e92ad5ffad None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:39.530340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 925/1851] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:39.537906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cdab9-521a-42a1-a110-73e92ad5ffad None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:39.545204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899e8d1-f2e3-4a7c-a2b2-e8bd730c86b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:39.546517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899e8d1-f2e3-4a7c-a2b2-e8bd730c86b6 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.546834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899e8d1-f2e3-4a7c-a2b2-e8bd730c86b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.547111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899e8d1-f2e3-4a7c-a2b2-e8bd730c86b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.548172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899e8d1-f2e3-4a7c-a2b2-e8bd730c86b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:39.550501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899e8d1-f2e3-4a7c-a2b2-e8bd730c86b6 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:39.701217 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 927/1852] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:39 2022] POST /identity/v3/projects => generated 369 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:39.706714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d58dde-a867-4f84-88e9-100f71d27e70 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:39.721035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d58dde-a867-4f84-88e9-100f71d27e70 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:39.822817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c18b4-ce89-4e1e-87d5-281fd58f4328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:39.823516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c18b4-ce89-4e1e-87d5-281fd58f4328 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.828338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c18b4-ce89-4e1e-87d5-281fd58f4328 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.828338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c18b4-ce89-4e1e-87d5-281fd58f4328 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.828338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c18b4-ce89-4e1e-87d5-281fd58f4328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:39.828956 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c18b4-ce89-4e1e-87d5-281fd58f4328 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:39.846949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:39.939946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 926/1853] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:39 2022] POST /identity/v3/users => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:39.941161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899e8d1-f2e3-4a7c-a2b2-e8bd730c86b6 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:39.953192 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899e8d1-f2e3-4a7c-a2b2-e8bd730c86b6 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:39.957285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:39.962906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90786eb-78d3-4a74-9abf-4588893e2b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:39.964593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90786eb-78d3-4a74-9abf-4588893e2b6e admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.964958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90786eb-78d3-4a74-9abf-4588893e2b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.965242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90786eb-78d3-4a74-9abf-4588893e2b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.966117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90786eb-78d3-4a74-9abf-4588893e2b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:39.968712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90786eb-78d3-4a74-9abf-4588893e2b6e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:39.985040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 928/1854] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:39.986268 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c18b4-ce89-4e1e-87d5-281fd58f4328 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:39.997586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c18b4-ce89-4e1e-87d5-281fd58f4328 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:40.006259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f10fd7-6bc9-4975-90e2-2b8fd52ef587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:40.007114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f10fd7-6bc9-4975-90e2-2b8fd52ef587 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.007497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f10fd7-6bc9-4975-90e2-2b8fd52ef587 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.007840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f10fd7-6bc9-4975-90e2-2b8fd52ef587 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:40.008653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f10fd7-6bc9-4975-90e2-2b8fd52ef587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:40.023471 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f10fd7-6bc9-4975-90e2-2b8fd52ef587 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:40.041129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 927/1855] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:40.045508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90786eb-78d3-4a74-9abf-4588893e2b6e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:40.062619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90786eb-78d3-4a74-9abf-4588893e2b6e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:40.063917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d662e-455c-4710-bace-ff5e928eb91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:40.064905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d662e-455c-4710-bace-ff5e928eb91e admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.065268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d662e-455c-4710-bace-ff5e928eb91e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.065620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d662e-455c-4710-bace-ff5e928eb91e admin admin] PATH_INFO: `/v3/projects/c8f2229f42f846d8b06ab10d6f7051e4/users/8e429c0359884f7bbc3f3e52fe76d557/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:40.082562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d662e-455c-4710-bace-ff5e928eb91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f2229f42f846d8b06ab10d6f7051e4, user_id=8e429c0359884f7bbc3f3e52fe76d557, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:40.085274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d662e-455c-4710-bace-ff5e928eb91e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:40.183524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 929/1856] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:40 2022] PUT /identity/v3/projects/c8f2229f42f846d8b06ab10d6f7051e4/users/8e429c0359884f7bbc3f3e52fe76d557/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:40.197463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5420ad3-c7d5-4446-97fc-cde9d89aab19 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.197759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5420ad3-c7d5-4446-97fc-cde9d89aab19 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.198039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5420ad3-c7d5-4446-97fc-cde9d89aab19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:40.229378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f5420ad3-c7d5-4446-97fc-cde9d89aab19 None None] MFA Rules not processed for user `8e429c0359884f7bbc3f3e52fe76d557`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:40.230644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f5420ad3-c7d5-4446-97fc-cde9d89aab19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:40.311024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 928/1857] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:40.315388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d662e-455c-4710-bace-ff5e928eb91e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:40.414416 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d662e-455c-4710-bace-ff5e928eb91e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:40.420434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0ec25-678b-4a61-af20-6ca310e96951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:40.421266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0ec25-678b-4a61-af20-6ca310e96951 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.422532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0ec25-678b-4a61-af20-6ca310e96951 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.423167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0ec25-678b-4a61-af20-6ca310e96951 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:40.424253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0ec25-678b-4a61-af20-6ca310e96951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:40.426413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0ec25-678b-4a61-af20-6ca310e96951 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:40.462654 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5420ad3-c7d5-4446-97fc-cde9d89aab19 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:40.464122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f5420ad3-c7d5-4446-97fc-cde9d89aab19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:40.483209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5420ad3-c7d5-4446-97fc-cde9d89aab19 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:40.496535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:40.497720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.498187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.499222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] PATH_INFO: `/v3/projects/397a7f96f3fc4680926ba227af78cb4d` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:40.501339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=397a7f96f3fc4680926ba227af78cb4d)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:40.508807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:40.532053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 930/1858] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:40 2022] POST /identity/v3/projects => generated 369 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:40.536798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0ec25-678b-4a61-af20-6ca310e96951 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:40.675405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0ec25-678b-4a61-af20-6ca310e96951 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:40.688215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '397a7f96f3fc4680926ba227af78cb4d', 'request_id': 'req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:40.697118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '397a7f96f3fc4680926ba227af78cb4d', 'request_id': 'req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:40.705932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '397a7f96f3fc4680926ba227af78cb4d', 'request_id': 'req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:40.796806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] The token cache is being invalidate because project 397a7f96f3fc4680926ba227af78cb4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:40.797768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:40.799227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:40.804964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 929/1859] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:40 2022] DELETE /identity/v3/projects/397a7f96f3fc4680926ba227af78cb4d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:40.804964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:40.821349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:40.850839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd56c55-2006-40a0-b69b-70bbb67ff609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:40.852177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd56c55-2006-40a0-b69b-70bbb67ff609 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.852465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd56c55-2006-40a0-b69b-70bbb67ff609 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.852742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd56c55-2006-40a0-b69b-70bbb67ff609 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:40.854963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd56c55-2006-40a0-b69b-70bbb67ff609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:40.861747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd56c55-2006-40a0-b69b-70bbb67ff609 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:40.866668 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79a131-5e04-4ada-a84c-3a1b5f48c1f0 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:40.881993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:40.886675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.886675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.886675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] PATH_INFO: `/v3/projects/9654aefbc1874980b8b5a7dc663817d8` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:40.888879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9654aefbc1874980b8b5a7dc663817d8)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:40.891775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:40.991241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 931/1860] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:40 2022] POST /identity/v3/users => generated 291 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:40.995146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153e481-1fc2-42a7-8eaf-d503cc9c09c7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.995146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153e481-1fc2-42a7-8eaf-d503cc9c09c7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.995146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153e481-1fc2-42a7-8eaf-d503cc9c09c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.045029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9654aefbc1874980b8b5a7dc663817d8', 'request_id': 'req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:41.061837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9654aefbc1874980b8b5a7dc663817d8', 'request_id': 'req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:41.062724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-7153e481-1fc2-42a7-8eaf-d503cc9c09c7 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:41.064173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7153e481-1fc2-42a7-8eaf-d503cc9c09c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.078807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9654aefbc1874980b8b5a7dc663817d8', 'request_id': 'req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:41.158628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 932/1861] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:41.159561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153e481-1fc2-42a7-8eaf-d503cc9c09c7 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.167307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7153e481-1fc2-42a7-8eaf-d503cc9c09c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.191516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] The token cache is being invalidate because project 9654aefbc1874980b8b5a7dc663817d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:41.191812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:41.192969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-30c4ba22-cdbc-4882-a6ae-ff62e23ea467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:41.200238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153e481-1fc2-42a7-8eaf-d503cc9c09c7 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.201734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 930/1862] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:40 2022] DELETE /identity/v3/projects/9654aefbc1874980b8b5a7dc663817d8 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:41.204030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf883e-c701-46ca-9645-57348a22e534 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.204328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf883e-c701-46ca-9645-57348a22e534 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.204660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf883e-c701-46ca-9645-57348a22e534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.211587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f7d88-577d-40a0-b5f3-c2dd835d4889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.212287 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f7d88-577d-40a0-b5f3-c2dd835d4889 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.213169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f7d88-577d-40a0-b5f3-c2dd835d4889 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.213470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f7d88-577d-40a0-b5f3-c2dd835d4889 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.214428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f7d88-577d-40a0-b5f3-c2dd835d4889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.219766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f7d88-577d-40a0-b5f3-c2dd835d4889 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.227717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-3abf883e-c701-46ca-9645-57348a22e534 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:41.230045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3abf883e-c701-46ca-9645-57348a22e534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.242029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 931/1863] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:41.243404 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 933/1864] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:41.243592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f199-67c2-4112-b896-d5db3273b2ae None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.244022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f199-67c2-4112-b896-d5db3273b2ae None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.244299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f199-67c2-4112-b896-d5db3273b2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.249806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f7d88-577d-40a0-b5f3-c2dd835d4889 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.266539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-8bb3f199-67c2-4112-b896-d5db3273b2ae None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:41.268077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8bb3f199-67c2-4112-b896-d5db3273b2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.272567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f7d88-577d-40a0-b5f3-c2dd835d4889 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.281537 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:41.289122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa065cd0-65f4-4556-a58a-c7a9d3dcf0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.289810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa065cd0-65f4-4556-a58a-c7a9d3dcf0ec admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.290120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa065cd0-65f4-4556-a58a-c7a9d3dcf0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.290401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa065cd0-65f4-4556-a58a-c7a9d3dcf0ec admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.291168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa065cd0-65f4-4556-a58a-c7a9d3dcf0ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:41.291482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa065cd0-65f4-4556-a58a-c7a9d3dcf0ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.293087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa065cd0-65f4-4556-a58a-c7a9d3dcf0ec admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.297778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 932/1865] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:41.313765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3f199-67c2-4112-b896-d5db3273b2ae None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.315394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8bb3f199-67c2-4112-b896-d5db3273b2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.324247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 934/1866] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:41.325459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c77db-45a3-4f51-b7e6-9b2e9acb5f20 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.325740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c77db-45a3-4f51-b7e6-9b2e9acb5f20 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.326022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c77db-45a3-4f51-b7e6-9b2e9acb5f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.331141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:41.353481 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3f199-67c2-4112-b896-d5db3273b2ae None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.365250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-854c77db-45a3-4f51-b7e6-9b2e9acb5f20 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:41.365495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eab75e-9425-4ced-b0a6-f6ec6d46c0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.366303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eab75e-9425-4ced-b0a6-f6ec6d46c0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.366620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eab75e-9425-4ced-b0a6-f6ec6d46c0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.366904 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eab75e-9425-4ced-b0a6-f6ec6d46c0f4 admin admin] PATH_INFO: `/v3/projects/93a0985851c44c4893f2a6976172993d/users/52b7b3857452458f8e1c207168ead7be/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.368438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-854c77db-45a3-4f51-b7e6-9b2e9acb5f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.386243 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 935/1867] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:41.390553 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c77db-45a3-4f51-b7e6-9b2e9acb5f20 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.408068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eab75e-9425-4ced-b0a6-f6ec6d46c0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a0985851c44c4893f2a6976172993d, user_id=52b7b3857452458f8e1c207168ead7be, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.410899 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c77db-45a3-4f51-b7e6-9b2e9acb5f20 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.411702 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eab75e-9425-4ced-b0a6-f6ec6d46c0f4 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.424431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb154f4-45e2-44b7-8add-89f864bde5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.425277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb154f4-45e2-44b7-8add-89f864bde5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.425820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb154f4-45e2-44b7-8add-89f864bde5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.426201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb154f4-45e2-44b7-8add-89f864bde5c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.427138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb154f4-45e2-44b7-8add-89f864bde5c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.429199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb154f4-45e2-44b7-8add-89f864bde5c3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.502733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 933/1868] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:41 2022] PUT /identity/v3/projects/93a0985851c44c4893f2a6976172993d/users/52b7b3857452458f8e1c207168ead7be/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:41.503657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eab75e-9425-4ced-b0a6-f6ec6d46c0f4 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.588782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 936/1869] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/projects => generated 371 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:41.589960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb154f4-45e2-44b7-8add-89f864bde5c3 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.600382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:41.641113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eab75e-9425-4ced-b0a6-f6ec6d46c0f4 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.761116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d5c7f-db88-4a29-89c5-be5127ddb718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.761801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d5c7f-db88-4a29-89c5-be5127ddb718 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.762083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d5c7f-db88-4a29-89c5-be5127ddb718 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.762387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d5c7f-db88-4a29-89c5-be5127ddb718 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.763816 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d5c7f-db88-4a29-89c5-be5127ddb718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:41.764171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d5c7f-db88-4a29-89c5-be5127ddb718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.769064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d5c7f-db88-4a29-89c5-be5127ddb718 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.807973 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 934/1870] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:41.809660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d5c7f-db88-4a29-89c5-be5127ddb718 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.812870 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb154f4-45e2-44b7-8add-89f864bde5c3 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.828495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed3059-3c34-43b2-9f82-c8c31e2a2330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.829197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed3059-3c34-43b2-9f82-c8c31e2a2330 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.829478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed3059-3c34-43b2-9f82-c8c31e2a2330 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.832250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed3059-3c34-43b2-9f82-c8c31e2a2330 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.832250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed3059-3c34-43b2-9f82-c8c31e2a2330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.835371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed3059-3c34-43b2-9f82-c8c31e2a2330 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.838895 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d5c7f-db88-4a29-89c5-be5127ddb718 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.848658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa468e0-4db5-40d5-9656-6bfd9d3bf433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.849348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa468e0-4db5-40d5-9656-6bfd9d3bf433 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.849627 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa468e0-4db5-40d5-9656-6bfd9d3bf433 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.849901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa468e0-4db5-40d5-9656-6bfd9d3bf433 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.850836 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa468e0-4db5-40d5-9656-6bfd9d3bf433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.852555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 937/1871] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:41.853007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa468e0-4db5-40d5-9656-6bfd9d3bf433 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.854194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed3059-3c34-43b2-9f82-c8c31e2a2330 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.867538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed3059-3c34-43b2-9f82-c8c31e2a2330 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.874002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee07bd0-1dad-458a-ac3c-8c20978ca458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.875739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee07bd0-1dad-458a-ac3c-8c20978ca458 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.876278 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee07bd0-1dad-458a-ac3c-8c20978ca458 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.876847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee07bd0-1dad-458a-ac3c-8c20978ca458 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.877804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee07bd0-1dad-458a-ac3c-8c20978ca458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.880378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee07bd0-1dad-458a-ac3c-8c20978ca458 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.949140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 935/1872] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/users => generated 286 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:41.949867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa468e0-4db5-40d5-9656-6bfd9d3bf433 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.962020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa468e0-4db5-40d5-9656-6bfd9d3bf433 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.974458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd24782-0748-48d3-a533-0aaae3c331f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.975410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd24782-0748-48d3-a533-0aaae3c331f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.975694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd24782-0748-48d3-a533-0aaae3c331f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.976331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd24782-0748-48d3-a533-0aaae3c331f4 admin admin] PATH_INFO: `/v3/projects/93a0985851c44c4893f2a6976172993d/users/52b7b3857452458f8e1c207168ead7be/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.979048 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd24782-0748-48d3-a533-0aaae3c331f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a0985851c44c4893f2a6976172993d, user_id=52b7b3857452458f8e1c207168ead7be, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.981582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd24782-0748-48d3-a533-0aaae3c331f4 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.994287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 936/1873] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:41 2022] PUT /identity/v3/projects/93a0985851c44c4893f2a6976172993d/users/52b7b3857452458f8e1c207168ead7be/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:42.004689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd24782-0748-48d3-a533-0aaae3c331f4 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.009935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 938/1874] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:42.014753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966e5b3-fd1e-487e-9544-b6eb3e91c0b5 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.015139 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966e5b3-fd1e-487e-9544-b6eb3e91c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.015415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966e5b3-fd1e-487e-9544-b6eb3e91c0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.059771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e966e5b3-fd1e-487e-9544-b6eb3e91c0b5 None None] MFA Rules not processed for user `52b7b3857452458f8e1c207168ead7be`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:42.061012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e966e5b3-fd1e-487e-9544-b6eb3e91c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:42.120500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd24782-0748-48d3-a533-0aaae3c331f4 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.161438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 939/1875] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:42 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:42.162165 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966e5b3-fd1e-487e-9544-b6eb3e91c0b5 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.254786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278475c-c361-4f55-aa6b-f1c985dbb3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.255821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278475c-c361-4f55-aa6b-f1c985dbb3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.256648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278475c-c361-4f55-aa6b-f1c985dbb3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.257080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278475c-c361-4f55-aa6b-f1c985dbb3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.258708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278475c-c361-4f55-aa6b-f1c985dbb3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.261428 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278475c-c361-4f55-aa6b-f1c985dbb3d3 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.282825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 937/1876] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:42.284533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278475c-c361-4f55-aa6b-f1c985dbb3d3 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.286876 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4278475c-c361-4f55-aa6b-f1c985dbb3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:42.292944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966e5b3-fd1e-487e-9544-b6eb3e91c0b5 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.307915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51715315-5938-4512-bb88-7c41c416efd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.309174 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51715315-5938-4512-bb88-7c41c416efd8 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.309674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51715315-5938-4512-bb88-7c41c416efd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.310131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51715315-5938-4512-bb88-7c41c416efd8 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.313820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51715315-5938-4512-bb88-7c41c416efd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.320350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:42.321364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51715315-5938-4512-bb88-7c41c416efd8 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.407474 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278475c-c361-4f55-aa6b-f1c985dbb3d3 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.412673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a947f25-dc8b-40ff-85a5-63d299ce3f00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.413359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a947f25-dc8b-40ff-85a5-63d299ce3f00 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.415445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a947f25-dc8b-40ff-85a5-63d299ce3f00 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.415445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a947f25-dc8b-40ff-85a5-63d299ce3f00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.430551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a947f25-dc8b-40ff-85a5-63d299ce3f00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.433410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a947f25-dc8b-40ff-85a5-63d299ce3f00 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.441142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 940/1877] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:42 2022] POST /identity/v3/users => generated 289 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:42.441802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51715315-5938-4512-bb88-7c41c416efd8 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.457436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:42.463904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51715315-5938-4512-bb88-7c41c416efd8 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.482061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 938/1878] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:42 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:42.483666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a947f25-dc8b-40ff-85a5-63d299ce3f00 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.499993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc061b-1557-45c7-a158-6270c9744b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.500955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc061b-1557-45c7-a158-6270c9744b38 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.501515 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc061b-1557-45c7-a158-6270c9744b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.501788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc061b-1557-45c7-a158-6270c9744b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.502555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc061b-1557-45c7-a158-6270c9744b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.504169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc061b-1557-45c7-a158-6270c9744b38 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.520322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a947f25-dc8b-40ff-85a5-63d299ce3f00 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.533232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 941/1879] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:42.535370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc061b-1557-45c7-a158-6270c9744b38 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.536400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f1fe8-e307-4f37-a9d2-cf0683f2dbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.537104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f1fe8-e307-4f37-a9d2-cf0683f2dbfa admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.537653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f1fe8-e307-4f37-a9d2-cf0683f2dbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.537936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f1fe8-e307-4f37-a9d2-cf0683f2dbfa admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.538701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f1fe8-e307-4f37-a9d2-cf0683f2dbfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.546697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f1fe8-e307-4f37-a9d2-cf0683f2dbfa admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.568992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc061b-1557-45c7-a158-6270c9744b38 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.575792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 939/1880] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:42.579188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e50b6-031d-49a4-a4a1-7f11246b7f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.579390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f1fe8-e307-4f37-a9d2-cf0683f2dbfa admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.580108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e50b6-031d-49a4-a4a1-7f11246b7f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.580505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e50b6-031d-49a4-a4a1-7f11246b7f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.580958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e50b6-031d-49a4-a4a1-7f11246b7f29 admin admin] PATH_INFO: `/v3/projects/2ba82caedf2b487e8bd25d3fed96c432/users/435672ec26f34b54858659ae32a2334e/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.594234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f1fe8-e307-4f37-a9d2-cf0683f2dbfa admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.602855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060247cc-994a-4411-a31c-103334ce9c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.604779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060247cc-994a-4411-a31c-103334ce9c84 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.605824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060247cc-994a-4411-a31c-103334ce9c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.606694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060247cc-994a-4411-a31c-103334ce9c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.607790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e50b6-031d-49a4-a4a1-7f11246b7f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba82caedf2b487e8bd25d3fed96c432, user_id=435672ec26f34b54858659ae32a2334e, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.608248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060247cc-994a-4411-a31c-103334ce9c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.610139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060247cc-994a-4411-a31c-103334ce9c84 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.610972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e50b6-031d-49a4-a4a1-7f11246b7f29 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.626194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 940/1881] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:42.639673 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060247cc-994a-4411-a31c-103334ce9c84 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.661193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060247cc-994a-4411-a31c-103334ce9c84 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.663528 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35faccf-dd67-4c4d-bb10-9b05d5d1f8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.664444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35faccf-dd67-4c4d-bb10-9b05d5d1f8da admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.664725 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35faccf-dd67-4c4d-bb10-9b05d5d1f8da admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.665441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35faccf-dd67-4c4d-bb10-9b05d5d1f8da admin admin] PATH_INFO: `/v3/projects/624427700ff7457186c8cdc33dab0c6d/users/28823304d92848f48c949564a6298c63/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.685106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35faccf-dd67-4c4d-bb10-9b05d5d1f8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=624427700ff7457186c8cdc33dab0c6d, user_id=28823304d92848f48c949564a6298c63, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.690604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35faccf-dd67-4c4d-bb10-9b05d5d1f8da admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.719579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 942/1882] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:42 2022] PUT /identity/v3/projects/2ba82caedf2b487e8bd25d3fed96c432/users/435672ec26f34b54858659ae32a2334e/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:42.721445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa9951-e4a9-447f-ac7f-cc61295e621f None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.721935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa9951-e4a9-447f-ac7f-cc61295e621f None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.723078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa9951-e4a9-447f-ac7f-cc61295e621f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.743130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-5bfa9951-e4a9-447f-ac7f-cc61295e621f None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:42.747512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5bfa9951-e4a9-447f-ac7f-cc61295e621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:42.807070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 941/1883] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:42 2022] PUT /identity/v3/projects/624427700ff7457186c8cdc33dab0c6d/users/28823304d92848f48c949564a6298c63/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:42.817377 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ff4a8-3dd8-4f40-9df1-8a5846aaf6b5 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.817989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ff4a8-3dd8-4f40-9df1-8a5846aaf6b5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.817989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ff4a8-3dd8-4f40-9df1-8a5846aaf6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.838815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 943/1884] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:42.860327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa9951-e4a9-447f-ac7f-cc61295e621f None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.863049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5bfa9951-e4a9-447f-ac7f-cc61295e621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:42.916563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-163ff4a8-3dd8-4f40-9df1-8a5846aaf6b5 None None] MFA Rules not processed for user `28823304d92848f48c949564a6298c63`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:42.922965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-163ff4a8-3dd8-4f40-9df1-8a5846aaf6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:42.930175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:43.016991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa9951-e4a9-447f-ac7f-cc61295e621f None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:43.043502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 942/1885] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:43.044755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ff4a8-3dd8-4f40-9df1-8a5846aaf6b5 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:43.046732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-163ff4a8-3dd8-4f40-9df1-8a5846aaf6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:43.104017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:43.108960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7109ae7-a2d3-42f7-b88a-04fac13743c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:43.109646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7109ae7-a2d3-42f7-b88a-04fac13743c3 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:43.109932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7109ae7-a2d3-42f7-b88a-04fac13743c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:43.110209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7109ae7-a2d3-42f7-b88a-04fac13743c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:43.132501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:43.232723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ff4a8-3dd8-4f40-9df1-8a5846aaf6b5 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:43.249909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f996886-dcc5-419f-9358-a2164e2c0bdc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e12361da5f849ee97607ca68ee6ef10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:43.252274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f996886-dcc5-419f-9358-a2164e2c0bdc service glance] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:43.252390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f996886-dcc5-419f-9358-a2164e2c0bdc service glance] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:43.252633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f996886-dcc5-419f-9358-a2164e2c0bdc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:43.264735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7109ae7-a2d3-42f7-b88a-04fac13743c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:43.266953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7109ae7-a2d3-42f7-b88a-04fac13743c3 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:43.272843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f996886-dcc5-419f-9358-a2164e2c0bdc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:43.275313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f996886-dcc5-419f-9358-a2164e2c0bdc service glance] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:43.291806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 943/1886] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:43.308711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 944/1887] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:45.215711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aee92d-e372-415c-a5d3-90563b6c730d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.232237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aee92d-e372-415c-a5d3-90563b6c730d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.232237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aee92d-e372-415c-a5d3-90563b6c730d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.268935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-03aee92d-e372-415c-a5d3-90563b6c730d None None] MFA Rules not processed for user `435672ec26f34b54858659ae32a2334e`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:45.270175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-03aee92d-e372-415c-a5d3-90563b6c730d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:45.362405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:45.398325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 944/1888] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:45.402224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7109ae7-a2d3-42f7-b88a-04fac13743c3 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:45.414620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7109ae7-a2d3-42f7-b88a-04fac13743c3 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:45.421146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b61b8b-69c6-45a9-b99b-a08c3f887465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:45.422225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b61b8b-69c6-45a9-b99b-a08c3f887465 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.422535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b61b8b-69c6-45a9-b99b-a08c3f887465 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.422975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b61b8b-69c6-45a9-b99b-a08c3f887465 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.426570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b61b8b-69c6-45a9-b99b-a08c3f887465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:45.426570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b61b8b-69c6-45a9-b99b-a08c3f887465 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:45.536493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 945/1889] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:45 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:45.539872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aee92d-e372-415c-a5d3-90563b6c730d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:45.551323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aee92d-e372-415c-a5d3-90563b6c730d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:45.669305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9840f-772e-4991-b429-f47f77c24413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:45.670031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9840f-772e-4991-b429-f47f77c24413 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.670316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9840f-772e-4991-b429-f47f77c24413 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.670625 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9840f-772e-4991-b429-f47f77c24413 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.671648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9840f-772e-4991-b429-f47f77c24413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:45.674085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9840f-772e-4991-b429-f47f77c24413 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:45.775720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 945/1890] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:45 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:45.779782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b61b8b-69c6-45a9-b99b-a08c3f887465 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:45.793109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b61b8b-69c6-45a9-b99b-a08c3f887465 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:45.800638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a059-dedf-437b-95fe-64d09baf2ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:45.801717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a059-dedf-437b-95fe-64d09baf2ecc admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.802229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a059-dedf-437b-95fe-64d09baf2ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.802713 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a059-dedf-437b-95fe-64d09baf2ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.803918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869a059-dedf-437b-95fe-64d09baf2ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:45.806418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869a059-dedf-437b-95fe-64d09baf2ecc admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:45.823156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 946/1891] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:45.827238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9840f-772e-4991-b429-f47f77c24413 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:45.840259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9840f-772e-4991-b429-f47f77c24413 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:45.845246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277314b1-979c-4876-ba9f-85e5a9aaad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:45.846293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277314b1-979c-4876-ba9f-85e5a9aaad1c admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.846647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277314b1-979c-4876-ba9f-85e5a9aaad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.846975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277314b1-979c-4876-ba9f-85e5a9aaad1c admin admin] PATH_INFO: `/v3/projects/4a5c8c262af448bea543d424b5980ec6/users/290d077cfbeb4fc8b33687b360e48b08/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.864013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277314b1-979c-4876-ba9f-85e5a9aaad1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5c8c262af448bea543d424b5980ec6, user_id=290d077cfbeb4fc8b33687b360e48b08, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:45.866741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277314b1-979c-4876-ba9f-85e5a9aaad1c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:45.939586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a059-dedf-437b-95fe-64d09baf2ecc admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:45.949615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 946/1892] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:45 2022] PUT /identity/v3/projects/4a5c8c262af448bea543d424b5980ec6/users/290d077cfbeb4fc8b33687b360e48b08/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:45.950318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a059-dedf-437b-95fe-64d09baf2ecc admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:45.954429 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277314b1-979c-4876-ba9f-85e5a9aaad1c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:46.006247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:46.016001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:46.016895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:46.017209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:46.020668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] PATH_INFO: `/v3/users/8e429c0359884f7bbc3f3e52fe76d557` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:46.020668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e429c0359884f7bbc3f3e52fe76d557)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:46.020874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277314b1-979c-4876-ba9f-85e5a9aaad1c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:46.020985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:46.025036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69ea05-aa7b-4862-b4d6-d98907f6979c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:46.025648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69ea05-aa7b-4862-b4d6-d98907f6979c admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:46.025938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69ea05-aa7b-4862-b4d6-d98907f6979c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:46.026219 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69ea05-aa7b-4862-b4d6-d98907f6979c admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:46.026998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69ea05-aa7b-4862-b4d6-d98907f6979c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:46.028627 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69ea05-aa7b-4862-b4d6-d98907f6979c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:46.038878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 947/1893] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:46.042903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69ea05-aa7b-4862-b4d6-d98907f6979c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:46.053564 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69ea05-aa7b-4862-b4d6-d98907f6979c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:46.059060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856215f-37bc-4c77-afc3-c786e49be1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:46.060006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856215f-37bc-4c77-afc3-c786e49be1b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:46.060333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856215f-37bc-4c77-afc3-c786e49be1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:46.060611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856215f-37bc-4c77-afc3-c786e49be1b4 admin admin] PATH_INFO: `/v3/projects/4a5c8c262af448bea543d424b5980ec6/users/290d077cfbeb4fc8b33687b360e48b08/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:46.063375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856215f-37bc-4c77-afc3-c786e49be1b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5c8c262af448bea543d424b5980ec6, user_id=290d077cfbeb4fc8b33687b360e48b08, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:46.066051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856215f-37bc-4c77-afc3-c786e49be1b4 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:46.076989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 948/1894] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:46 2022] PUT /identity/v3/projects/4a5c8c262af448bea543d424b5980ec6/users/290d077cfbeb4fc8b33687b360e48b08/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:46.250219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e429c0359884f7bbc3f3e52fe76d557'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:46.257863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e429c0359884f7bbc3f3e52fe76d557'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:46.258587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e429c0359884f7bbc3f3e52fe76d557'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:46.354038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e429c0359884f7bbc3f3e52fe76d557'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:46.354806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e429c0359884f7bbc3f3e52fe76d557'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:46.374119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 947/1895] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:45 2022] DELETE /identity/v3/users/8e429c0359884f7bbc3f3e52fe76d557 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:46.378829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f675ce-855f-4267-b775-0aecc52d00a0 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:46.379334 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f675ce-855f-4267-b775-0aecc52d00a0 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:46.379424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f675ce-855f-4267-b775-0aecc52d00a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:46.396967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-53f675ce-855f-4267-b775-0aecc52d00a0 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:46.398198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-53f675ce-855f-4267-b775-0aecc52d00a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:46.436479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:46.481642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 949/1896] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:46.492823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:46.494378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:46.552462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8359e72-82ff-4a71-b41c-0d8f35b71339 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:46.555927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388173fc-c4ab-4793-90c3-7cdc7ceee498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:46.564747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388173fc-c4ab-4793-90c3-7cdc7ceee498 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:46.564747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388173fc-c4ab-4793-90c3-7cdc7ceee498 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:46.564747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388173fc-c4ab-4793-90c3-7cdc7ceee498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:46.565993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388173fc-c4ab-4793-90c3-7cdc7ceee498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:46.567455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388173fc-c4ab-4793-90c3-7cdc7ceee498 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:46.576821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 948/1897] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:47.144306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f675ce-855f-4267-b775-0aecc52d00a0 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:47.145796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-53f675ce-855f-4267-b775-0aecc52d00a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:47.158397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f675ce-855f-4267-b775-0aecc52d00a0 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:47.163423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:47.165103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:47.165103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:47.165103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] PATH_INFO: `/v3/projects/c8f2229f42f846d8b06ab10d6f7051e4` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:47.166115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f2229f42f846d8b06ab10d6f7051e4)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:47.168399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:47.303212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f2229f42f846d8b06ab10d6f7051e4', 'request_id': 'req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.313704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f2229f42f846d8b06ab10d6f7051e4', 'request_id': 'req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.320305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f2229f42f846d8b06ab10d6f7051e4', 'request_id': 'req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.393569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] The token cache is being invalidate because project c8f2229f42f846d8b06ab10d6f7051e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:47.394194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.394511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21dd7f4a-e982-4f26-afbd-e1eddc8552f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.396105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 950/1898] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:47 2022] DELETE /identity/v3/projects/c8f2229f42f846d8b06ab10d6f7051e4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:47.400236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388173fc-c4ab-4793-90c3-7cdc7ceee498 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:47.412942 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388173fc-c4ab-4793-90c3-7cdc7ceee498 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:47.469317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a294d-8260-4bd1-999e-7800f2520d39 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:47.469691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a294d-8260-4bd1-999e-7800f2520d39 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:47.469983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a294d-8260-4bd1-999e-7800f2520d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:47.485932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:47.486864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:47.487148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:47.487427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] PATH_INFO: `/v3/users/52b7b3857452458f8e1c207168ead7be` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:47.488862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52b7b3857452458f8e1c207168ead7be)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:47.490841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:47.503470 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f99a294d-8260-4bd1-999e-7800f2520d39 None None] MFA Rules not processed for user `290d077cfbeb4fc8b33687b360e48b08`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:47.504891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f99a294d-8260-4bd1-999e-7800f2520d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:47.506567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:47.583204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 951/1899] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:47.594704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a294d-8260-4bd1-999e-7800f2520d39 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:47.596103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f99a294d-8260-4bd1-999e-7800f2520d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:47.645140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:47.655900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a294d-8260-4bd1-999e-7800f2520d39 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:47.659512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab3261-0875-4dca-94e7-6948b59976b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:47.660157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab3261-0875-4dca-94e7-6948b59976b7 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:47.660437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab3261-0875-4dca-94e7-6948b59976b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:47.660712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab3261-0875-4dca-94e7-6948b59976b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:47.662767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-59ab3261-0875-4dca-94e7-6948b59976b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:47.734914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab3261-0875-4dca-94e7-6948b59976b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:47.736388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab3261-0875-4dca-94e7-6948b59976b7 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:47.742857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52b7b3857452458f8e1c207168ead7be'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.746061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 952/1900] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:47.753565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52b7b3857452458f8e1c207168ead7be'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.758624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52b7b3857452458f8e1c207168ead7be'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.967536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52b7b3857452458f8e1c207168ead7be'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.967536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-29dc3a62-e6c0-460d-90c0-8a7a64cbfde5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52b7b3857452458f8e1c207168ead7be'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.967536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 949/1901] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:47 2022] DELETE /identity/v3/users/52b7b3857452458f8e1c207168ead7be => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:48.616615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab3261-0875-4dca-94e7-6948b59976b7 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:48.616615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-59ab3261-0875-4dca-94e7-6948b59976b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:48.638003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab3261-0875-4dca-94e7-6948b59976b7 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:48.704031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:48.704932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:48.705212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:48.705483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] PATH_INFO: `/v3/projects/93a0985851c44c4893f2a6976172993d` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:48.706828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a0985851c44c4893f2a6976172993d)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:48.708829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:48.855884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a0985851c44c4893f2a6976172993d', 'request_id': 'req-e78d601f-baab-413b-b456-70c730c65c2b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:48.863500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a0985851c44c4893f2a6976172993d', 'request_id': 'req-e78d601f-baab-413b-b456-70c730c65c2b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:48.869182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a0985851c44c4893f2a6976172993d', 'request_id': 'req-e78d601f-baab-413b-b456-70c730c65c2b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:48.948603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] The token cache is being invalidate because project 93a0985851c44c4893f2a6976172993d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:48.948965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:48.949594 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e78d601f-baab-413b-b456-70c730c65c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:48.951237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 953/1902] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:48 2022] DELETE /identity/v3/projects/93a0985851c44c4893f2a6976172993d => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:48.958578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337e01f-2ee7-4133-9bb7-ca2a3be12afb None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:48.959080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337e01f-2ee7-4133-9bb7-ca2a3be12afb None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:48.959469 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337e01f-2ee7-4133-9bb7-ca2a3be12afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:48.983292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8337e01f-2ee7-4133-9bb7-ca2a3be12afb None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:48.985145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8337e01f-2ee7-4133-9bb7-ca2a3be12afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:49.044596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 950/1903] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:49.053217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4ee5b-5f99-4d4d-95e8-f63cff78bda9 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.053508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4ee5b-5f99-4d4d-95e8-f63cff78bda9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.053812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4ee5b-5f99-4d4d-95e8-f63cff78bda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.073199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-08c4ee5b-5f99-4d4d-95e8-f63cff78bda9 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:49.074408 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-08c4ee5b-5f99-4d4d-95e8-f63cff78bda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:49.082944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 954/1904] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:49.095333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337e01f-2ee7-4133-9bb7-ca2a3be12afb None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:49.105129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337e01f-2ee7-4133-9bb7-ca2a3be12afb None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:49.109975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b95987-73b0-4a1c-b570-a60abe9777d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:49.110671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b95987-73b0-4a1c-b570-a60abe9777d9 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.110959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b95987-73b0-4a1c-b570-a60abe9777d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.111240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b95987-73b0-4a1c-b570-a60abe9777d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.111972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b95987-73b0-4a1c-b570-a60abe9777d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:49.112420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b95987-73b0-4a1c-b570-a60abe9777d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:49.114078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b95987-73b0-4a1c-b570-a60abe9777d9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:49.125802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 951/1905] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:59:49.135602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4ee5b-5f99-4d4d-95e8-f63cff78bda9 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:49.146774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4ee5b-5f99-4d4d-95e8-f63cff78bda9 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:49.151490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0e406-0b5c-43e8-aba1-03e481b87663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:49.152189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0e406-0b5c-43e8-aba1-03e481b87663 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.152474 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0e406-0b5c-43e8-aba1-03e481b87663 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.152753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0e406-0b5c-43e8-aba1-03e481b87663 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.153470 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0e406-0b5c-43e8-aba1-03e481b87663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:49.155480 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0e406-0b5c-43e8-aba1-03e481b87663 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:49.246889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:49.256826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 955/1906] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:49 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:49.258205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b95987-73b0-4a1c-b570-a60abe9777d9 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:49.269169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b95987-73b0-4a1c-b570-a60abe9777d9 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:49.332434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b3b8d-2e29-45d0-a4be-561ce9c89174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:49.332672 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b3b8d-2e29-45d0-a4be-561ce9c89174 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.332837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b3b8d-2e29-45d0-a4be-561ce9c89174 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.333051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b3b8d-2e29-45d0-a4be-561ce9c89174 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.333891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b3b8d-2e29-45d0-a4be-561ce9c89174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:49.335828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b3b8d-2e29-45d0-a4be-561ce9c89174 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:49.420415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 952/1907] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:49 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:49.421764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0e406-0b5c-43e8-aba1-03e481b87663 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:49.433454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0e406-0b5c-43e8-aba1-03e481b87663 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:49.442233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdea94-9ea2-4703-827a-b3e56563e5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:49.442233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdea94-9ea2-4703-827a-b3e56563e5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.442233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdea94-9ea2-4703-827a-b3e56563e5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.442233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdea94-9ea2-4703-827a-b3e56563e5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.442642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdea94-9ea2-4703-827a-b3e56563e5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:49.442642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdea94-9ea2-4703-827a-b3e56563e5d8 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:49.454300 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 956/1908] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:49.464155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b3b8d-2e29-45d0-a4be-561ce9c89174 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:49.474604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b3b8d-2e29-45d0-a4be-561ce9c89174 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:49.479258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0c222-09f2-4d0b-8286-31e616ea694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:49.484675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0c222-09f2-4d0b-8286-31e616ea694d admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.484675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0c222-09f2-4d0b-8286-31e616ea694d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.484675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0c222-09f2-4d0b-8286-31e616ea694d admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.484675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0c222-09f2-4d0b-8286-31e616ea694d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:49.484675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0c222-09f2-4d0b-8286-31e616ea694d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:49.485507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:49.494654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 953/1909] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:49.499062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdea94-9ea2-4703-827a-b3e56563e5d8 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:49.513250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdea94-9ea2-4703-827a-b3e56563e5d8 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:49.519103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec818e-7afc-4059-a28d-a1bb0b72df7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:49.520005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec818e-7afc-4059-a28d-a1bb0b72df7e admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.520287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec818e-7afc-4059-a28d-a1bb0b72df7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.520567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec818e-7afc-4059-a28d-a1bb0b72df7e admin admin] PATH_INFO: `/v3/projects/99cd188c0bfd4a82b20f50e159a0b4b3/users/06af2f651cad47f595c8615c12926cc9/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.536910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec818e-7afc-4059-a28d-a1bb0b72df7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99cd188c0bfd4a82b20f50e159a0b4b3, user_id=06af2f651cad47f595c8615c12926cc9, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:49.540036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec818e-7afc-4059-a28d-a1bb0b72df7e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:49.605882 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 957/1910] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:49 2022] PUT /identity/v3/projects/99cd188c0bfd4a82b20f50e159a0b4b3/users/06af2f651cad47f595c8615c12926cc9/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:49.616234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab964c3a-37b7-49d5-aadf-04fc89270837 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.616522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab964c3a-37b7-49d5-aadf-04fc89270837 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.616797 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab964c3a-37b7-49d5-aadf-04fc89270837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.644370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ab964c3a-37b7-49d5-aadf-04fc89270837 None None] MFA Rules not processed for user `06af2f651cad47f595c8615c12926cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:49.645596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ab964c3a-37b7-49d5-aadf-04fc89270837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:49.713168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 954/1911] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:49.717608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec818e-7afc-4059-a28d-a1bb0b72df7e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:49.790243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec818e-7afc-4059-a28d-a1bb0b72df7e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:49.795473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276d6b0-ec40-43d7-b791-6e80b55433d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:49.796142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276d6b0-ec40-43d7-b791-6e80b55433d9 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.796486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276d6b0-ec40-43d7-b791-6e80b55433d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.798506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276d6b0-ec40-43d7-b791-6e80b55433d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.798506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276d6b0-ec40-43d7-b791-6e80b55433d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:49.802719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276d6b0-ec40-43d7-b791-6e80b55433d9 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:49.915343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 958/1912] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:49 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:49.919594 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab964c3a-37b7-49d5-aadf-04fc89270837 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:49.932896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab964c3a-37b7-49d5-aadf-04fc89270837 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:49.954352 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276d6b0-ec40-43d7-b791-6e80b55433d9 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:49.956068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8276d6b0-ec40-43d7-b791-6e80b55433d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:50.005392 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dd9a8-ad1e-426d-8497-055905cb3c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.006080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dd9a8-ad1e-426d-8497-055905cb3c58 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.006366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dd9a8-ad1e-426d-8497-055905cb3c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.006676 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dd9a8-ad1e-426d-8497-055905cb3c58 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.007368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dd9a8-ad1e-426d-8497-055905cb3c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.009285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dd9a8-ad1e-426d-8497-055905cb3c58 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:50.014391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276d6b0-ec40-43d7-b791-6e80b55433d9 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:50.018799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1136177-8586-46b1-b901-56dce308145b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.019487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1136177-8586-46b1-b901-56dce308145b service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.019771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1136177-8586-46b1-b901-56dce308145b service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.020046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1136177-8586-46b1-b901-56dce308145b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.021848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d1136177-8586-46b1-b901-56dce308145b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:50.088589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1136177-8586-46b1-b901-56dce308145b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.090114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1136177-8586-46b1-b901-56dce308145b service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:50.095729 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:50.100986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 959/1913] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:50.114505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 955/1914] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:49 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:50.118512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1136177-8586-46b1-b901-56dce308145b service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:50.129516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1136177-8586-46b1-b901-56dce308145b service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:50.134139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49824711-0297-41ac-8e25-a8e76954a12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.134823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49824711-0297-41ac-8e25-a8e76954a12c admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.135104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49824711-0297-41ac-8e25-a8e76954a12c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.135407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49824711-0297-41ac-8e25-a8e76954a12c admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.136139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49824711-0297-41ac-8e25-a8e76954a12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.138775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49824711-0297-41ac-8e25-a8e76954a12c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:50.150443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 960/1915] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:50.154915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dd9a8-ad1e-426d-8497-055905cb3c58 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:50.165658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dd9a8-ad1e-426d-8497-055905cb3c58 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:50.170651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e2f88-23fc-47e8-aaff-5dd7f7313e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.171583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e2f88-23fc-47e8-aaff-5dd7f7313e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.171864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e2f88-23fc-47e8-aaff-5dd7f7313e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.172147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e2f88-23fc-47e8-aaff-5dd7f7313e60 admin admin] PATH_INFO: `/v3/projects/8f34fedf1b0c40eca3bd39fb8444920e/users/30d9b992b1d0425d89cc06d8c07a64d1/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.190717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e2f88-23fc-47e8-aaff-5dd7f7313e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f34fedf1b0c40eca3bd39fb8444920e, user_id=30d9b992b1d0425d89cc06d8c07a64d1, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.193426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e2f88-23fc-47e8-aaff-5dd7f7313e60 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:50.258742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 956/1916] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:50 2022] PUT /identity/v3/projects/8f34fedf1b0c40eca3bd39fb8444920e/users/30d9b992b1d0425d89cc06d8c07a64d1/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:50.262950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49824711-0297-41ac-8e25-a8e76954a12c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:50.274464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49824711-0297-41ac-8e25-a8e76954a12c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:50.335826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbfea7-dcdf-4b8c-9e21-a561c1ebc439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.336493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfea7-dcdf-4b8c-9e21-a561c1ebc439 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.336774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfea7-dcdf-4b8c-9e21-a561c1ebc439 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.337047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfea7-dcdf-4b8c-9e21-a561c1ebc439 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.337764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbfea7-dcdf-4b8c-9e21-a561c1ebc439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.339420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbfea7-dcdf-4b8c-9e21-a561c1ebc439 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:50.349227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 961/1917] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:50.353587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e2f88-23fc-47e8-aaff-5dd7f7313e60 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:50.365594 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e2f88-23fc-47e8-aaff-5dd7f7313e60 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:50.371570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd08069-b8ba-49f8-be81-f10323c3eb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.372492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd08069-b8ba-49f8-be81-f10323c3eb14 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.372774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd08069-b8ba-49f8-be81-f10323c3eb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.373052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd08069-b8ba-49f8-be81-f10323c3eb14 admin admin] PATH_INFO: `/v3/projects/8f34fedf1b0c40eca3bd39fb8444920e/users/30d9b992b1d0425d89cc06d8c07a64d1/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.375203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd08069-b8ba-49f8-be81-f10323c3eb14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f34fedf1b0c40eca3bd39fb8444920e, user_id=30d9b992b1d0425d89cc06d8c07a64d1, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.378464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd08069-b8ba-49f8-be81-f10323c3eb14 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:50.392885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 957/1918] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:50 2022] PUT /identity/v3/projects/8f34fedf1b0c40eca3bd39fb8444920e/users/30d9b992b1d0425d89cc06d8c07a64d1/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:50.405432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e156a-313b-4329-969b-f9147dc097f0 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.405912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e156a-313b-4329-969b-f9147dc097f0 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.406342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e156a-313b-4329-969b-f9147dc097f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.435356 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-c35e156a-313b-4329-969b-f9147dc097f0 None None] MFA Rules not processed for user `30d9b992b1d0425d89cc06d8c07a64d1`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:50.436396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c35e156a-313b-4329-969b-f9147dc097f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:50.533340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 962/1919] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:50.548426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd08069-b8ba-49f8-be81-f10323c3eb14 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:50.550123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bcd08069-b8ba-49f8-be81-f10323c3eb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:50.597080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:50.611123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd08069-b8ba-49f8-be81-f10323c3eb14 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:50.615173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f396285-44b3-4894-af41-0fe3f13c2449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.615865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f396285-44b3-4894-af41-0fe3f13c2449 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.616142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f396285-44b3-4894-af41-0fe3f13c2449 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.616415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f396285-44b3-4894-af41-0fe3f13c2449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.629376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f396285-44b3-4894-af41-0fe3f13c2449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.630987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f396285-44b3-4894-af41-0fe3f13c2449 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:50.643316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 958/1920] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:50.747553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e156a-313b-4329-969b-f9147dc097f0 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:50.756919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e156a-313b-4329-969b-f9147dc097f0 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:50.760638 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60919c7-e3db-4024-ae2f-52ac3e73bdb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.761293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60919c7-e3db-4024-ae2f-52ac3e73bdb6 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.761606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60919c7-e3db-4024-ae2f-52ac3e73bdb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.761911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60919c7-e3db-4024-ae2f-52ac3e73bdb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.819998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60919c7-e3db-4024-ae2f-52ac3e73bdb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.821517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60919c7-e3db-4024-ae2f-52ac3e73bdb6 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:50.831145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 963/1921] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:53.860832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f396285-44b3-4894-af41-0fe3f13c2449 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:53.924943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f396285-44b3-4894-af41-0fe3f13c2449 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:53.929650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:53.930560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:53.930844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:53.931121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] PATH_INFO: `/v3/users/06af2f651cad47f595c8615c12926cc9` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:53.932333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] RBAC: Authorizing `identity:delete_user(user_id=06af2f651cad47f595c8615c12926cc9)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:53.934323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:54.121740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06af2f651cad47f595c8615c12926cc9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:54.133524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06af2f651cad47f595c8615c12926cc9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:54.134251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06af2f651cad47f595c8615c12926cc9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:54.234443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06af2f651cad47f595c8615c12926cc9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:54.235823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06af2f651cad47f595c8615c12926cc9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:54.257903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 959/1922] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:53 2022] DELETE /identity/v3/users/06af2f651cad47f595c8615c12926cc9 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:54.262630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d907883-f1e0-4875-9944-ef6e7a046301 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:54.263478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d907883-f1e0-4875-9944-ef6e7a046301 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:54.267255 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d907883-f1e0-4875-9944-ef6e7a046301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:54.282278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-7d907883-f1e0-4875-9944-ef6e7a046301 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:54.283566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7d907883-f1e0-4875-9944-ef6e7a046301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:54.288092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:54.354434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 964/1923] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:54.371477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:54.373152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:54.466339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e06cb-7baa-46cd-9e07-0c155874b69f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:54.466339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b6e07-d03b-495e-86e6-8137f55f8e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:54.466339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6e07-d03b-495e-86e6-8137f55f8e6e service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:54.466339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6e07-d03b-495e-86e6-8137f55f8e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:54.466693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6e07-d03b-495e-86e6-8137f55f8e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:54.476591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b6e07-d03b-495e-86e6-8137f55f8e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:54.478079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b6e07-d03b-495e-86e6-8137f55f8e6e service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:54.489542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 960/1924] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:54.961618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d907883-f1e0-4875-9944-ef6e7a046301 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:54.963228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7d907883-f1e0-4875-9944-ef6e7a046301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:54.979159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d907883-f1e0-4875-9944-ef6e7a046301 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:54.983900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:54.984772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:54.985046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:54.985317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] PATH_INFO: `/v3/projects/99cd188c0bfd4a82b20f50e159a0b4b3` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:54.987621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99cd188c0bfd4a82b20f50e159a0b4b3)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:54.989516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:55.126356 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99cd188c0bfd4a82b20f50e159a0b4b3', 'request_id': 'req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:55.133674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99cd188c0bfd4a82b20f50e159a0b4b3', 'request_id': 'req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:55.139481 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99cd188c0bfd4a82b20f50e159a0b4b3', 'request_id': 'req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:55.268575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] The token cache is being invalidate because project 99cd188c0bfd4a82b20f50e159a0b4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:55.269234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:55.269582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:55.271268 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 965/1925] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:54 2022] DELETE /identity/v3/projects/99cd188c0bfd4a82b20f50e159a0b4b3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:55.275227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b6e07-d03b-495e-86e6-8137f55f8e6e service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:55.283263 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:55.287668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b6e07-d03b-495e-86e6-8137f55f8e6e service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:55.366993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:55.368184 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:55.368471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:55.368748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] PATH_INFO: `/v3/users/30d9b992b1d0425d89cc06d8c07a64d1` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:55.370221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30d9b992b1d0425d89cc06d8c07a64d1)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:55.372616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:55.625277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30d9b992b1d0425d89cc06d8c07a64d1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:55.633758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30d9b992b1d0425d89cc06d8c07a64d1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:55.634466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30d9b992b1d0425d89cc06d8c07a64d1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:55.708615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30d9b992b1d0425d89cc06d8c07a64d1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:55.709260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3b3c0ecf-8b6c-463c-b6ee-5cb8e001c708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30d9b992b1d0425d89cc06d8c07a64d1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:55.734222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 961/1926] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:55 2022] DELETE /identity/v3/users/30d9b992b1d0425d89cc06d8c07a64d1 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:56.190314 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:56.191831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:56.205249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cffa5-6ab7-4e1b-a15b-c30d6cc51db1 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.262388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.263229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.263508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.263781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] PATH_INFO: `/v3/projects/8f34fedf1b0c40eca3bd39fb8444920e` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.265044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f34fedf1b0c40eca3bd39fb8444920e)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:56.266990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:56.347762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:56.358462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f34fedf1b0c40eca3bd39fb8444920e', 'request_id': 'req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:56.365452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f34fedf1b0c40eca3bd39fb8444920e', 'request_id': 'req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:56.371737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f34fedf1b0c40eca3bd39fb8444920e', 'request_id': 'req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:56.479538 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] The token cache is being invalidate because project 8f34fedf1b0c40eca3bd39fb8444920e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:56.479849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:56.480518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-cf38b4a9-2b2d-421c-b6e7-2cb38bf0c251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:56.482134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 966/1927] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:56 2022] DELETE /identity/v3/projects/8f34fedf1b0c40eca3bd39fb8444920e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:56.487054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3244bf-7ba2-42ae-88e7-76b6759bd156 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.487546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3244bf-7ba2-42ae-88e7-76b6759bd156 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.487637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3244bf-7ba2-42ae-88e7-76b6759bd156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.505834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-1c3244bf-7ba2-42ae-88e7-76b6759bd156 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:56.507090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1c3244bf-7ba2-42ae-88e7-76b6759bd156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:56.569493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 962/1928] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:56.579762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b90b8b-d553-43b3-a965-552b6e079eb4 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.580348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b90b8b-d553-43b3-a965-552b6e079eb4 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.581030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b90b8b-d553-43b3-a965-552b6e079eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.602426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f8b90b8b-d553-43b3-a965-552b6e079eb4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:56.603716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f8b90b8b-d553-43b3-a965-552b6e079eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:56.612030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 967/1929] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:56.625249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3244bf-7ba2-42ae-88e7-76b6759bd156 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:56.626583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3244bf-7ba2-42ae-88e7-76b6759bd156 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.633787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb3845-b45c-4b58-9699-e5848cbb9667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.633787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb3845-b45c-4b58-9699-e5848cbb9667 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.633787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb3845-b45c-4b58-9699-e5848cbb9667 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.633787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb3845-b45c-4b58-9699-e5848cbb9667 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.634089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb3845-b45c-4b58-9699-e5848cbb9667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:56.634089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb3845-b45c-4b58-9699-e5848cbb9667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:56.635298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb3845-b45c-4b58-9699-e5848cbb9667 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:56.648203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 963/1930] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:59:56.652615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b90b8b-d553-43b3-a965-552b6e079eb4 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:56.663061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b90b8b-d553-43b3-a965-552b6e079eb4 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.668559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5abbaa-50f1-4dd1-8d9d-20869694c18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.669037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5abbaa-50f1-4dd1-8d9d-20869694c18c admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.669321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5abbaa-50f1-4dd1-8d9d-20869694c18c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.669602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5abbaa-50f1-4dd1-8d9d-20869694c18c admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.670304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5abbaa-50f1-4dd1-8d9d-20869694c18c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:56.672325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5abbaa-50f1-4dd1-8d9d-20869694c18c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:56.738549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 968/1931] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:56 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:56.742721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb3845-b45c-4b58-9699-e5848cbb9667 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:56.753337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb3845-b45c-4b58-9699-e5848cbb9667 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.761440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:56.826800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907fb8e-5f0e-4741-a832-4d3fbbc844b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.826800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907fb8e-5f0e-4741-a832-4d3fbbc844b9 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.826800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907fb8e-5f0e-4741-a832-4d3fbbc844b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.826800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907fb8e-5f0e-4741-a832-4d3fbbc844b9 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.827166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907fb8e-5f0e-4741-a832-4d3fbbc844b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:56.827166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907fb8e-5f0e-4741-a832-4d3fbbc844b9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:56.907377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 964/1932] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:56 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:56.908473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5abbaa-50f1-4dd1-8d9d-20869694c18c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:56.918994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5abbaa-50f1-4dd1-8d9d-20869694c18c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.924053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb8132-1a3a-42e5-b20b-83e94ea54e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.924792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb8132-1a3a-42e5-b20b-83e94ea54e72 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.925072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb8132-1a3a-42e5-b20b-83e94ea54e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.925347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb8132-1a3a-42e5-b20b-83e94ea54e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.926235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb8132-1a3a-42e5-b20b-83e94ea54e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:56.928063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb8132-1a3a-42e5-b20b-83e94ea54e72 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:56.939755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 969/1933] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:56.943818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907fb8e-5f0e-4741-a832-4d3fbbc844b9 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:56.954381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907fb8e-5f0e-4741-a832-4d3fbbc844b9 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.959115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45efe7c-8e1e-4bc5-b635-df0a4e08d855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.959776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45efe7c-8e1e-4bc5-b635-df0a4e08d855 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.960064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45efe7c-8e1e-4bc5-b635-df0a4e08d855 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.960343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45efe7c-8e1e-4bc5-b635-df0a4e08d855 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.961099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45efe7c-8e1e-4bc5-b635-df0a4e08d855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:56.962709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45efe7c-8e1e-4bc5-b635-df0a4e08d855 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:56.972671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 965/1934] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:56.977416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb8132-1a3a-42e5-b20b-83e94ea54e72 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:56.987240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb8132-1a3a-42e5-b20b-83e94ea54e72 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.991897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386a663-0311-4f01-9628-f34183066465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.992769 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386a663-0311-4f01-9628-f34183066465 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.993052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386a663-0311-4f01-9628-f34183066465 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.993329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386a663-0311-4f01-9628-f34183066465 admin admin] PATH_INFO: `/v3/projects/f526375156da4ec4b45563bf250cc464/users/ae04513bc55d4e20be71a8fbf910841f/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:57.009124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386a663-0311-4f01-9628-f34183066465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f526375156da4ec4b45563bf250cc464, user_id=ae04513bc55d4e20be71a8fbf910841f, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:57.011879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386a663-0311-4f01-9628-f34183066465 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:57.091169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 970/1935] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:56 2022] PUT /identity/v3/projects/f526375156da4ec4b45563bf250cc464/users/ae04513bc55d4e20be71a8fbf910841f/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:57.102880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e48906-90bb-4a5a-849e-105d8cdf3d9c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:57.103177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e48906-90bb-4a5a-849e-105d8cdf3d9c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:57.103488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e48906-90bb-4a5a-849e-105d8cdf3d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:57.133388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a7e48906-90bb-4a5a-849e-105d8cdf3d9c None None] MFA Rules not processed for user `ae04513bc55d4e20be71a8fbf910841f`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:57.134681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a7e48906-90bb-4a5a-849e-105d8cdf3d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:57.223772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 966/1936] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:57.235446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386a663-0311-4f01-9628-f34183066465 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:57.237099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1386a663-0311-4f01-9628-f34183066465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:57.308559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386a663-0311-4f01-9628-f34183066465 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:57.312250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8434c-c14c-45af-a40e-c215a2398bb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:57.312997 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8434c-c14c-45af-a40e-c215a2398bb0 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:57.313276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8434c-c14c-45af-a40e-c215a2398bb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:57.313555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8434c-c14c-45af-a40e-c215a2398bb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:57.322127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8434c-c14c-45af-a40e-c215a2398bb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:57.323618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8434c-c14c-45af-a40e-c215a2398bb0 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:57.333624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 971/1937] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:57.352922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e48906-90bb-4a5a-849e-105d8cdf3d9c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:57.422783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e48906-90bb-4a5a-849e-105d8cdf3d9c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:57.426336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:57.427385 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:57.427800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:57.428089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] PATH_INFO: `/v3/users/ae04513bc55d4e20be71a8fbf910841f` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:57.429376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae04513bc55d4e20be71a8fbf910841f)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:57.431384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:57.456048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:57.643503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae04513bc55d4e20be71a8fbf910841f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:57.650880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae04513bc55d4e20be71a8fbf910841f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:57.656221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae04513bc55d4e20be71a8fbf910841f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:57.743150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae04513bc55d4e20be71a8fbf910841f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:57.749797 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae04513bc55d4e20be71a8fbf910841f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:57.771072 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 967/1938] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:57 2022] DELETE /identity/v3/users/ae04513bc55d4e20be71a8fbf910841f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:57.775734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155e880-0689-4c54-9e06-837ea863dc41 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:57.776192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155e880-0689-4c54-9e06-837ea863dc41 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:57.776309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155e880-0689-4c54-9e06-837ea863dc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:57.794383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-2155e880-0689-4c54-9e06-837ea863dc41 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:57.795632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2155e880-0689-4c54-9e06-837ea863dc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:57.897237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 972/1939] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:57.907109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:57.908668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:57.974535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090ad7d-a4de-46e1-a10f-ad7a351d9f0f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:57.978563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdf4ba-94c4-4c61-ab2f-d4bf30e00747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:57.979461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdf4ba-94c4-4c61-ab2f-d4bf30e00747 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:57.979609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdf4ba-94c4-4c61-ab2f-d4bf30e00747 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:57.979811 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdf4ba-94c4-4c61-ab2f-d4bf30e00747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:57.990372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdf4ba-94c4-4c61-ab2f-d4bf30e00747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:57.991853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdf4ba-94c4-4c61-ab2f-d4bf30e00747 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:58.003206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 968/1940] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:58.487354 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155e880-0689-4c54-9e06-837ea863dc41 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:58.489007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2155e880-0689-4c54-9e06-837ea863dc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:58.496837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:58.510848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155e880-0689-4c54-9e06-837ea863dc41 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:58.531329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:58.532523 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:58.532763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:58.533042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] PATH_INFO: `/v3/projects/f526375156da4ec4b45563bf250cc464` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:58.534352 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f526375156da4ec4b45563bf250cc464)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:58.539577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:58.710139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f526375156da4ec4b45563bf250cc464', 'request_id': 'req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:58.711725 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f526375156da4ec4b45563bf250cc464', 'request_id': 'req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:58.720148 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f526375156da4ec4b45563bf250cc464', 'request_id': 'req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:58.783104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] The token cache is being invalidate because project f526375156da4ec4b45563bf250cc464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:58.783774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:58.784217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2cbd6cfc-9fd1-4902-97b1-ece4b8067b40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:58.785835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 973/1941] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:58 2022] DELETE /identity/v3/projects/f526375156da4ec4b45563bf250cc464 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:58.791386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efbe1e-2bca-4b26-9f4c-e42a22c1b7ef None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:58.791933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efbe1e-2bca-4b26-9f4c-e42a22c1b7ef None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:58.791933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efbe1e-2bca-4b26-9f4c-e42a22c1b7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:58.812383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-35efbe1e-2bca-4b26-9f4c-e42a22c1b7ef None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:58.813658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-35efbe1e-2bca-4b26-9f4c-e42a22c1b7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:58.890046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 969/1942] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:58.899026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eecb9-b7fe-4a5d-a3a5-b7ee118d4582 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:58.899634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eecb9-b7fe-4a5d-a3a5-b7ee118d4582 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:58.900800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eecb9-b7fe-4a5d-a3a5-b7ee118d4582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.068294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-b58eecb9-b7fe-4a5d-a3a5-b7ee118d4582 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:59.068294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b58eecb9-b7fe-4a5d-a3a5-b7ee118d4582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:59.068294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 974/1943] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:59.068564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efbe1e-2bca-4b26-9f4c-e42a22c1b7ef None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:59.068564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efbe1e-2bca-4b26-9f4c-e42a22c1b7ef None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:59.068564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f88bd8-51f7-44fe-aedf-36f2f5d46840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:59.068564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f88bd8-51f7-44fe-aedf-36f2f5d46840 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:59.068907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f88bd8-51f7-44fe-aedf-36f2f5d46840 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:59.068907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f88bd8-51f7-44fe-aedf-36f2f5d46840 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.068907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f88bd8-51f7-44fe-aedf-36f2f5d46840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:59.068907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f88bd8-51f7-44fe-aedf-36f2f5d46840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:59.068907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f88bd8-51f7-44fe-aedf-36f2f5d46840 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:59.068907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 970/1944] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:59.069340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58eecb9-b7fe-4a5d-a3a5-b7ee118d4582 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:59.069340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58eecb9-b7fe-4a5d-a3a5-b7ee118d4582 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:59.069340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbf3ab-c1f6-42dc-ac58-31191c741c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:59.069340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbf3ab-c1f6-42dc-ac58-31191c741c2f admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:59.069621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbf3ab-c1f6-42dc-ac58-31191c741c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:59.069621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbf3ab-c1f6-42dc-ac58-31191c741c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.069621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbf3ab-c1f6-42dc-ac58-31191c741c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:59.069621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbf3ab-c1f6-42dc-ac58-31191c741c2f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:59.127249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 975/1945] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:58 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:59:59.133028 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f88bd8-51f7-44fe-aedf-36f2f5d46840 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:59.146353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f88bd8-51f7-44fe-aedf-36f2f5d46840 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:59.174074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:59.210009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362540a-2154-4efd-aa22-95a206b3dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:59.214917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362540a-2154-4efd-aa22-95a206b3dbe6 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:59.214917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362540a-2154-4efd-aa22-95a206b3dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:59.214917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362540a-2154-4efd-aa22-95a206b3dbe6 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.214917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362540a-2154-4efd-aa22-95a206b3dbe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:59.214917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362540a-2154-4efd-aa22-95a206b3dbe6 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:59.327852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 971/1946] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:59 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:59.333219 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbf3ab-c1f6-42dc-ac58-31191c741c2f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:59.350698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbf3ab-c1f6-42dc-ac58-31191c741c2f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:59.376839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eda22c-bbc6-4b14-9f0b-46a2936ebf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:59.377957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eda22c-bbc6-4b14-9f0b-46a2936ebf7f admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:59.378225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eda22c-bbc6-4b14-9f0b-46a2936ebf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:59.378655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eda22c-bbc6-4b14-9f0b-46a2936ebf7f admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.379506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eda22c-bbc6-4b14-9f0b-46a2936ebf7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:59.383069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eda22c-bbc6-4b14-9f0b-46a2936ebf7f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:59.398898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 976/1947] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:59.402783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362540a-2154-4efd-aa22-95a206b3dbe6 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:59.417536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362540a-2154-4efd-aa22-95a206b3dbe6 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:59.422940 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a23cf-1001-46f4-9188-2359995b8f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:59.424124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a23cf-1001-46f4-9188-2359995b8f12 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:59.424585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a23cf-1001-46f4-9188-2359995b8f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:59.424976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a23cf-1001-46f4-9188-2359995b8f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.425992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a23cf-1001-46f4-9188-2359995b8f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:59.428045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a23cf-1001-46f4-9188-2359995b8f12 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:59.442499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 972/1948] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:59.444338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eda22c-bbc6-4b14-9f0b-46a2936ebf7f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:59.454671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eda22c-bbc6-4b14-9f0b-46a2936ebf7f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:59.459138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8542a-3f7c-48b9-9083-a282b7ee0b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:59.460025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8542a-3f7c-48b9-9083-a282b7ee0b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:59.460302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8542a-3f7c-48b9-9083-a282b7ee0b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:59.460576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8542a-3f7c-48b9-9083-a282b7ee0b53 admin admin] PATH_INFO: `/v3/projects/2bb6418e0f1441599929c77141079855/users/80be150f20ae4a6fb4d365059d336103/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.476361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8542a-3f7c-48b9-9083-a282b7ee0b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb6418e0f1441599929c77141079855, user_id=80be150f20ae4a6fb4d365059d336103, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:59.479185 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8542a-3f7c-48b9-9083-a282b7ee0b53 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:59.533718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 977/1949] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:59 2022] PUT /identity/v3/projects/2bb6418e0f1441599929c77141079855/users/80be150f20ae4a6fb4d365059d336103/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:59.544732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcbe0a-c319-4e9d-b78d-2f5c381a6d38 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:59.545018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcbe0a-c319-4e9d-b78d-2f5c381a6d38 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:59.545339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcbe0a-c319-4e9d-b78d-2f5c381a6d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.577669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a3fcbe0a-c319-4e9d-b78d-2f5c381a6d38 None None] MFA Rules not processed for user `80be150f20ae4a6fb4d365059d336103`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:59.579702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a3fcbe0a-c319-4e9d-b78d-2f5c381a6d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:59.663946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 973/1950] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 05:59:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:59.675677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8542a-3f7c-48b9-9083-a282b7ee0b53 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:59.677403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-04b8542a-3f7c-48b9-9083-a282b7ee0b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:59.747458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8542a-3f7c-48b9-9083-a282b7ee0b53 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:59.752189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807e125-f873-4b45-b387-6407860fbd6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:59.752858 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807e125-f873-4b45-b387-6407860fbd6a service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:59.753142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807e125-f873-4b45-b387-6407860fbd6a service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:59.753419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807e125-f873-4b45-b387-6407860fbd6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.763728 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807e125-f873-4b45-b387-6407860fbd6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:59.769395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807e125-f873-4b45-b387-6407860fbd6a service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:59.790269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 978/1951] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:01.432268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2ac9f-f08d-4d48-977f-4936d1eb89fe None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:01.433327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2ac9f-f08d-4d48-977f-4936d1eb89fe None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:01.433671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2ac9f-f08d-4d48-977f-4936d1eb89fe None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:01.435337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 974/1952] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 06:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:00:01.450260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ecd82-124c-42c5-a056-c123b6c24e68 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:01.451508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ecd82-124c-42c5-a056-c123b6c24e68 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:01.452068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ecd82-124c-42c5-a056-c123b6c24e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:01.473627 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-548ecd82-124c-42c5-a056-c123b6c24e68 None None] MFA Rules not processed for user `dbfcf2e915c049f08a9e41c555401a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:01.476285 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-548ecd82-124c-42c5-a056-c123b6c24e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:01.516628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:01.566860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 979/1953] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 06:00:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:01.581241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2ac9f-f08d-4d48-977f-4936d1eb89fe None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:01.583074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f5d2ac9f-f08d-4d48-977f-4936d1eb89fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:01.604186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ecd82-124c-42c5-a056-c123b6c24e68 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:01.605884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-548ecd82-124c-42c5-a056-c123b6c24e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:01.614835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2ac9f-f08d-4d48-977f-4936d1eb89fe None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:01.622320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a30b9-42a4-4fc1-a5fd-46afffa30824 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:01.623257 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a30b9-42a4-4fc1-a5fd-46afffa30824 service placement] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:01.623541 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a30b9-42a4-4fc1-a5fd-46afffa30824 service placement] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:01.623819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a30b9-42a4-4fc1-a5fd-46afffa30824 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:01.635011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a30b9-42a4-4fc1-a5fd-46afffa30824 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:01.636608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a30b9-42a4-4fc1-a5fd-46afffa30824 service placement] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:01.648542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 975/1954] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:01.676097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ecd82-124c-42c5-a056-c123b6c24e68 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:01.681191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:01.682109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:01.682582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:01.682928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] PATH_INFO: `/v3/users/3bc10b641e34490a83bc587262f4c61a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:01.684513 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bc10b641e34490a83bc587262f4c61a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:01.687474 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:01.859344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bc10b641e34490a83bc587262f4c61a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:01.866175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bc10b641e34490a83bc587262f4c61a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:01.959697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bc10b641e34490a83bc587262f4c61a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:01.960378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bc10b641e34490a83bc587262f4c61a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:01.961014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a4396b4b-8d7e-4645-90c4-f88117347a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bc10b641e34490a83bc587262f4c61a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:01.979600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 980/1955] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:01 2022] DELETE /identity/v3/users/3bc10b641e34490a83bc587262f4c61a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:02.266677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a30b9-42a4-4fc1-a5fd-46afffa30824 service placement] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:02.277533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a30b9-42a4-4fc1-a5fd-46afffa30824 service placement] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:02.338860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:02.339722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.340003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.340279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] PATH_INFO: `/v3/projects/bb20b5f0088f4a1c84f17ddb49d851a7` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.341481 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb20b5f0088f4a1c84f17ddb49d851a7)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:02.343421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:02.418288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:02.434880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb20b5f0088f4a1c84f17ddb49d851a7', 'request_id': 'req-2fae09d4-e2a3-4356-b566-1c933871e3af'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:02.442125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb20b5f0088f4a1c84f17ddb49d851a7', 'request_id': 'req-2fae09d4-e2a3-4356-b566-1c933871e3af'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:02.447839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb20b5f0088f4a1c84f17ddb49d851a7', 'request_id': 'req-2fae09d4-e2a3-4356-b566-1c933871e3af'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:02.502371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] The token cache is being invalidate because project bb20b5f0088f4a1c84f17ddb49d851a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:02.502690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:02.503333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2fae09d4-e2a3-4356-b566-1c933871e3af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:02.504879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 976/1956] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:02 2022] DELETE /identity/v3/projects/bb20b5f0088f4a1c84f17ddb49d851a7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:02.509801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27991a9-3d34-44a8-95f4-bbe59940895c None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.510325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27991a9-3d34-44a8-95f4-bbe59940895c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.510424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27991a9-3d34-44a8-95f4-bbe59940895c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.527883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a27991a9-3d34-44a8-95f4-bbe59940895c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:02.529083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a27991a9-3d34-44a8-95f4-bbe59940895c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:02.564311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:02.589486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 981/1957] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:02.598140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f58fb7-f290-4d4f-aeef-f09eb0dc934d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.598536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f58fb7-f290-4d4f-aeef-f09eb0dc934d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.598818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f58fb7-f290-4d4f-aeef-f09eb0dc934d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.617609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-02f58fb7-f290-4d4f-aeef-f09eb0dc934d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:02.618848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-02f58fb7-f290-4d4f-aeef-f09eb0dc934d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:02.627534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 977/1958] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:02.631854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27991a9-3d34-44a8-95f4-bbe59940895c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:02.641980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27991a9-3d34-44a8-95f4-bbe59940895c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:02.646612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f935a5-bb16-4626-aa48-efa23e4cbc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:02.647542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f935a5-bb16-4626-aa48-efa23e4cbc91 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.647829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f935a5-bb16-4626-aa48-efa23e4cbc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.648105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f935a5-bb16-4626-aa48-efa23e4cbc91 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.648837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f935a5-bb16-4626-aa48-efa23e4cbc91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:02.649142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f935a5-bb16-4626-aa48-efa23e4cbc91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:02.651045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f935a5-bb16-4626-aa48-efa23e4cbc91 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:02.664172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 982/1959] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:02.668574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f58fb7-f290-4d4f-aeef-f09eb0dc934d None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:02.678319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f58fb7-f290-4d4f-aeef-f09eb0dc934d None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:02.683095 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45713c-9db5-44ba-9c52-f6e1b44b1fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:02.683761 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45713c-9db5-44ba-9c52-f6e1b44b1fcd admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.684043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45713c-9db5-44ba-9c52-f6e1b44b1fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.684319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45713c-9db5-44ba-9c52-f6e1b44b1fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.685015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45713c-9db5-44ba-9c52-f6e1b44b1fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:02.686952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45713c-9db5-44ba-9c52-f6e1b44b1fcd admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:02.772700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 978/1960] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:02 2022] POST /identity/v3/projects => generated 357 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:02.776711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f935a5-bb16-4626-aa48-efa23e4cbc91 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:02.787825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f935a5-bb16-4626-aa48-efa23e4cbc91 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:02.844645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067be6c-da1e-4f82-be2c-3fe6cf186c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:02.845306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067be6c-da1e-4f82-be2c-3fe6cf186c33 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.845585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067be6c-da1e-4f82-be2c-3fe6cf186c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.845854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067be6c-da1e-4f82-be2c-3fe6cf186c33 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.846553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067be6c-da1e-4f82-be2c-3fe6cf186c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:02.848462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067be6c-da1e-4f82-be2c-3fe6cf186c33 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:02.948506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 983/1961] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:02 2022] POST /identity/v3/users => generated 279 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:02.952478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45713c-9db5-44ba-9c52-f6e1b44b1fcd admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:02.963416 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45713c-9db5-44ba-9c52-f6e1b44b1fcd admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:02.968120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de73f0-f997-4c90-8996-ff2e482f1545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:02.968824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de73f0-f997-4c90-8996-ff2e482f1545 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.969107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de73f0-f997-4c90-8996-ff2e482f1545 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.969384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de73f0-f997-4c90-8996-ff2e482f1545 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.970115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de73f0-f997-4c90-8996-ff2e482f1545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:02.971773 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de73f0-f997-4c90-8996-ff2e482f1545 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:02.982215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 979/1962] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:02.985840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067be6c-da1e-4f82-be2c-3fe6cf186c33 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:02.995701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067be6c-da1e-4f82-be2c-3fe6cf186c33 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:03.000813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88f158-4ff4-4fe2-bdf2-f83866172e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:03.001458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88f158-4ff4-4fe2-bdf2-f83866172e6e admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.001739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88f158-4ff4-4fe2-bdf2-f83866172e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.002009 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88f158-4ff4-4fe2-bdf2-f83866172e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.002758 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88f158-4ff4-4fe2-bdf2-f83866172e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:03.004375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88f158-4ff4-4fe2-bdf2-f83866172e6e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:03.014097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 984/1963] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:03.018732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de73f0-f997-4c90-8996-ff2e482f1545 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:03.028703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de73f0-f997-4c90-8996-ff2e482f1545 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:03.033930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc974f1-f201-4d39-b0f9-b52929b95f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:03.034835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc974f1-f201-4d39-b0f9-b52929b95f1e admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.035117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc974f1-f201-4d39-b0f9-b52929b95f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.035395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc974f1-f201-4d39-b0f9-b52929b95f1e admin admin] PATH_INFO: `/v3/projects/4bb8a8a686844154baecdf57a634c395/users/84379615d1a54acd8147960c57a391dc/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.050393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc974f1-f201-4d39-b0f9-b52929b95f1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb8a8a686844154baecdf57a634c395, user_id=84379615d1a54acd8147960c57a391dc, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:03.053138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc974f1-f201-4d39-b0f9-b52929b95f1e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:03.154049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 980/1964] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:03 2022] PUT /identity/v3/projects/4bb8a8a686844154baecdf57a634c395/users/84379615d1a54acd8147960c57a391dc/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:03.161074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27576cf9-4f73-454a-a6e1-9b15e09eb29d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.161251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27576cf9-4f73-454a-a6e1-9b15e09eb29d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.161530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27576cf9-4f73-454a-a6e1-9b15e09eb29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.188331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-27576cf9-4f73-454a-a6e1-9b15e09eb29d None None] MFA Rules not processed for user `84379615d1a54acd8147960c57a391dc`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:03.189536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-27576cf9-4f73-454a-a6e1-9b15e09eb29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:03.253647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 985/1965] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:03 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:03.258029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc974f1-f201-4d39-b0f9-b52929b95f1e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:03.320668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc974f1-f201-4d39-b0f9-b52929b95f1e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:03.325331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561ec6a-93d0-4232-92ce-fb67cf0092ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:03.326036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561ec6a-93d0-4232-92ce-fb67cf0092ab admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.326345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561ec6a-93d0-4232-92ce-fb67cf0092ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.326659 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561ec6a-93d0-4232-92ce-fb67cf0092ab admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.327386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561ec6a-93d0-4232-92ce-fb67cf0092ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:03.329812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561ec6a-93d0-4232-92ce-fb67cf0092ab admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:03.412794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 981/1966] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:03 2022] POST /identity/v3/projects => generated 353 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:03.416602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27576cf9-4f73-454a-a6e1-9b15e09eb29d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:03.427298 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27576cf9-4f73-454a-a6e1-9b15e09eb29d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:03.459280 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:03.482767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb5191-7d93-4919-8cd1-0707bf928cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:03.483465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb5191-7d93-4919-8cd1-0707bf928cbe admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.483755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb5191-7d93-4919-8cd1-0707bf928cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.484041 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb5191-7d93-4919-8cd1-0707bf928cbe admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.484756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb5191-7d93-4919-8cd1-0707bf928cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:03.486726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb5191-7d93-4919-8cd1-0707bf928cbe admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:03.566896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 986/1967] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:03 2022] POST /identity/v3/users => generated 283 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:03.571223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561ec6a-93d0-4232-92ce-fb67cf0092ab admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:03.582306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561ec6a-93d0-4232-92ce-fb67cf0092ab admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:03.587083 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6ab32-3929-4475-8df1-380ad41327e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:03.587788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6ab32-3929-4475-8df1-380ad41327e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.588076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6ab32-3929-4475-8df1-380ad41327e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.588373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6ab32-3929-4475-8df1-380ad41327e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.589106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6ab32-3929-4475-8df1-380ad41327e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:03.590879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6ab32-3929-4475-8df1-380ad41327e1 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:03.601627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 982/1968] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:03.605788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb5191-7d93-4919-8cd1-0707bf928cbe admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:03.615810 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb5191-7d93-4919-8cd1-0707bf928cbe admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:03.620590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42ade1-30b9-4646-8987-243130d33336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:03.621485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42ade1-30b9-4646-8987-243130d33336 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.621766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42ade1-30b9-4646-8987-243130d33336 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.622043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42ade1-30b9-4646-8987-243130d33336 admin admin] PATH_INFO: `/v3/projects/62ef20196ac649f6971e3776af606d3b/users/913fd2c237c64170b128b18a928d1caa/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.637312 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42ade1-30b9-4646-8987-243130d33336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ef20196ac649f6971e3776af606d3b, user_id=913fd2c237c64170b128b18a928d1caa, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:03.640002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42ade1-30b9-4646-8987-243130d33336 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:03.748091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 987/1969] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:03 2022] PUT /identity/v3/projects/62ef20196ac649f6971e3776af606d3b/users/913fd2c237c64170b128b18a928d1caa/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:03.752658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6ab32-3929-4475-8df1-380ad41327e1 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:03.762982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6ab32-3929-4475-8df1-380ad41327e1 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:03.783679 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:03.820623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e7ccd-0bb0-414d-b9c4-db740ee1e319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:03.821297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7ccd-0bb0-414d-b9c4-db740ee1e319 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.821578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7ccd-0bb0-414d-b9c4-db740ee1e319 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.821854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7ccd-0bb0-414d-b9c4-db740ee1e319 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.822602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e7ccd-0bb0-414d-b9c4-db740ee1e319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:03.824189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e7ccd-0bb0-414d-b9c4-db740ee1e319 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:03.834486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 983/1970] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:03.838499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42ade1-30b9-4646-8987-243130d33336 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:03.852064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42ade1-30b9-4646-8987-243130d33336 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:03.853912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e7f1c-7051-4083-952b-bd4422f3e8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:03.854828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e7f1c-7051-4083-952b-bd4422f3e8b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.855111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e7f1c-7051-4083-952b-bd4422f3e8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.855431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e7f1c-7051-4083-952b-bd4422f3e8b7 admin admin] PATH_INFO: `/v3/projects/62ef20196ac649f6971e3776af606d3b/users/913fd2c237c64170b128b18a928d1caa/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.857467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e7f1c-7051-4083-952b-bd4422f3e8b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ef20196ac649f6971e3776af606d3b, user_id=913fd2c237c64170b128b18a928d1caa, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:03.861044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e7f1c-7051-4083-952b-bd4422f3e8b7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:03.872050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 988/1971] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:03 2022] PUT /identity/v3/projects/62ef20196ac649f6971e3776af606d3b/users/913fd2c237c64170b128b18a928d1caa/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:03.882795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2551a3-0fba-4a9e-9bb1-356f2c5fee6c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.883083 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2551a3-0fba-4a9e-9bb1-356f2c5fee6c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.883390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2551a3-0fba-4a9e-9bb1-356f2c5fee6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.920517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-bc2551a3-0fba-4a9e-9bb1-356f2c5fee6c None None] MFA Rules not processed for user `913fd2c237c64170b128b18a928d1caa`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:03.921731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bc2551a3-0fba-4a9e-9bb1-356f2c5fee6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:03.991611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 984/1972] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:03 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:04.003586 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e7f1c-7051-4083-952b-bd4422f3e8b7 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:04.005259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8c9e7f1c-7051-4083-952b-bd4422f3e8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:04.061697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e7f1c-7051-4083-952b-bd4422f3e8b7 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:04.065863 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa5c7d-3df7-4c33-952b-8112fadd545d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:04.066536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa5c7d-3df7-4c33-952b-8112fadd545d service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:04.066822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa5c7d-3df7-4c33-952b-8112fadd545d service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:04.067100 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa5c7d-3df7-4c33-952b-8112fadd545d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:04.076001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa5c7d-3df7-4c33-952b-8112fadd545d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:04.077437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa5c7d-3df7-4c33-952b-8112fadd545d service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:04.086751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 989/1973] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:04 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:04.264526 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2551a3-0fba-4a9e-9bb1-356f2c5fee6c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:04.265961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bc2551a3-0fba-4a9e-9bb1-356f2c5fee6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:04.277860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2551a3-0fba-4a9e-9bb1-356f2c5fee6c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:04.281379 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e4a42-189f-45b2-aeb6-6948c28a3fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:04.282573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4a42-189f-45b2-aeb6-6948c28a3fbd service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:04.282857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4a42-189f-45b2-aeb6-6948c28a3fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:04.283131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4a42-189f-45b2-aeb6-6948c28a3fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:04.338819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e4a42-189f-45b2-aeb6-6948c28a3fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:04.340215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e4a42-189f-45b2-aeb6-6948c28a3fbd service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:04.349614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 985/1974] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:04 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:12.490269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa5c7d-3df7-4c33-952b-8112fadd545d service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:12.491693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-14aa5c7d-3df7-4c33-952b-8112fadd545d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:12.559683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa5c7d-3df7-4c33-952b-8112fadd545d service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:12.565337 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:12.566226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.566534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.566816 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] PATH_INFO: `/v3/users/28823304d92848f48c949564a6298c63` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.568154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28823304d92848f48c949564a6298c63)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:12.570093 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:12.761189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:12.773812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28823304d92848f48c949564a6298c63'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:12.784488 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28823304d92848f48c949564a6298c63'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:12.965438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28823304d92848f48c949564a6298c63'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:12.966144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28823304d92848f48c949564a6298c63'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:12.966815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28823304d92848f48c949564a6298c63'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:12.988548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 990/1975] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:12 2022] DELETE /identity/v3/users/28823304d92848f48c949564a6298c63 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:12.993031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6913d-1936-4922-bb3d-0f78544282d6 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.993502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6913d-1936-4922-bb3d-0f78544282d6 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.993580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6913d-1936-4922-bb3d-0f78544282d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:13.013962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2dc6913d-1936-4922-bb3d-0f78544282d6 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:13.015817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2dc6913d-1936-4922-bb3d-0f78544282d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:13.069446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:13.088997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 986/1976] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:13.100516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:13.102050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:13.192543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3576aaf-f6a5-4e13-9f53-e530d8552f62 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:13.196232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97b581-9359-4fe5-b848-10634f15f6f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:13.196886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97b581-9359-4fe5-b848-10634f15f6f5 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:13.197174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97b581-9359-4fe5-b848-10634f15f6f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:13.197504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97b581-9359-4fe5-b848-10634f15f6f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:13.207708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97b581-9359-4fe5-b848-10634f15f6f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:13.209208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97b581-9359-4fe5-b848-10634f15f6f5 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:13.221494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 991/1977] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:13.629541 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6913d-1936-4922-bb3d-0f78544282d6 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:13.631035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2dc6913d-1936-4922-bb3d-0f78544282d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:13.645437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6913d-1936-4922-bb3d-0f78544282d6 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:13.651819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:13.652894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:13.653336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:13.653711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] PATH_INFO: `/v3/projects/624427700ff7457186c8cdc33dab0c6d` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:13.655167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=624427700ff7457186c8cdc33dab0c6d)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:13.658685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:13.784753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624427700ff7457186c8cdc33dab0c6d', 'request_id': 'req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:13.791924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624427700ff7457186c8cdc33dab0c6d', 'request_id': 'req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:13.797748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624427700ff7457186c8cdc33dab0c6d', 'request_id': 'req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:13.914614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] The token cache is being invalidate because project 624427700ff7457186c8cdc33dab0c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:13.914909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:13.915559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b51124b3-7c1b-44cd-927e-91d3c0d07d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:13.917159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 987/1978] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:13 2022] DELETE /identity/v3/projects/624427700ff7457186c8cdc33dab0c6d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:13.922560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd33300-9dfd-4097-a1f1-ae0a5e5deb31 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:13.923038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd33300-9dfd-4097-a1f1-ae0a5e5deb31 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:13.923227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd33300-9dfd-4097-a1f1-ae0a5e5deb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:13.941875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9bd33300-9dfd-4097-a1f1-ae0a5e5deb31 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:13.943147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9bd33300-9dfd-4097-a1f1-ae0a5e5deb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:14.004659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 992/1979] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:14.012366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf446e7-51ae-4e5d-8597-1704c3cb9b18 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.012851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf446e7-51ae-4e5d-8597-1704c3cb9b18 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.012989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf446e7-51ae-4e5d-8597-1704c3cb9b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.031463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-caf446e7-51ae-4e5d-8597-1704c3cb9b18 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:14.032697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-caf446e7-51ae-4e5d-8597-1704c3cb9b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:14.040780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 988/1980] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:14.045100 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd33300-9dfd-4097-a1f1-ae0a5e5deb31 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:14.054777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd33300-9dfd-4097-a1f1-ae0a5e5deb31 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:14.059831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe04539-6b8d-4f0c-9030-b885d2b407f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:14.060530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe04539-6b8d-4f0c-9030-b885d2b407f8 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.060826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe04539-6b8d-4f0c-9030-b885d2b407f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.061123 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe04539-6b8d-4f0c-9030-b885d2b407f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.061845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe04539-6b8d-4f0c-9030-b885d2b407f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:14.062157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe04539-6b8d-4f0c-9030-b885d2b407f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:14.063872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe04539-6b8d-4f0c-9030-b885d2b407f8 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:14.077021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 993/1981] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:14.081262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf446e7-51ae-4e5d-8597-1704c3cb9b18 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:14.088008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe04539-6b8d-4f0c-9030-b885d2b407f8 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:14.090084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6fe04539-6b8d-4f0c-9030-b885d2b407f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:14.091778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf446e7-51ae-4e5d-8597-1704c3cb9b18 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:14.096948 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66c2a8-bfac-411b-8773-56f0972a0226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:14.098569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66c2a8-bfac-411b-8773-56f0972a0226 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.098955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66c2a8-bfac-411b-8773-56f0972a0226 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.099993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66c2a8-bfac-411b-8773-56f0972a0226 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.100865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66c2a8-bfac-411b-8773-56f0972a0226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:14.103175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66c2a8-bfac-411b-8773-56f0972a0226 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:14.103541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe04539-6b8d-4f0c-9030-b885d2b407f8 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:14.108832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:14.109714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.109996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.110272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] PATH_INFO: `/v3/users/435672ec26f34b54858659ae32a2334e` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.111575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=435672ec26f34b54858659ae32a2334e)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:14.113565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:14.178671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 989/1982] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:14 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:14.182702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66c2a8-bfac-411b-8773-56f0972a0226 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:14.250401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66c2a8-bfac-411b-8773-56f0972a0226 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:14.255648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca9a49-0a3f-46ac-b3d8-030e43dc1f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:14.256259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca9a49-0a3f-46ac-b3d8-030e43dc1f22 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.256566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca9a49-0a3f-46ac-b3d8-030e43dc1f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.256846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca9a49-0a3f-46ac-b3d8-030e43dc1f22 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.257565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca9a49-0a3f-46ac-b3d8-030e43dc1f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:14.259573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca9a49-0a3f-46ac-b3d8-030e43dc1f22 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:14.269877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:14.281144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '435672ec26f34b54858659ae32a2334e'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:14.295618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '435672ec26f34b54858659ae32a2334e'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:14.346352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 990/1983] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:14 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:14.347323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca9a49-0a3f-46ac-b3d8-030e43dc1f22 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:14.358215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca9a49-0a3f-46ac-b3d8-030e43dc1f22 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:14.364756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e214c-b45a-47a7-8474-5e8b063ba117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:14.365619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e214c-b45a-47a7-8474-5e8b063ba117 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.366043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e214c-b45a-47a7-8474-5e8b063ba117 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.382316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e214c-b45a-47a7-8474-5e8b063ba117 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.383251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e214c-b45a-47a7-8474-5e8b063ba117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:14.384909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e214c-b45a-47a7-8474-5e8b063ba117 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:14.386375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '435672ec26f34b54858659ae32a2334e'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:14.387655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '435672ec26f34b54858659ae32a2334e'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:14.388961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '435672ec26f34b54858659ae32a2334e'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:14.396857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 991/1984] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:14.400802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e214c-b45a-47a7-8474-5e8b063ba117 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:14.402589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1a4e214c-b45a-47a7-8474-5e8b063ba117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:14.410437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 994/1985] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:14 2022] DELETE /identity/v3/users/435672ec26f34b54858659ae32a2334e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:14.421438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e214c-b45a-47a7-8474-5e8b063ba117 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:14.436922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:14.483883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd971b4-6ee5-4098-b6df-7a51f511bf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:14.485130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd971b4-6ee5-4098-b6df-7a51f511bf9f admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.485130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd971b4-6ee5-4098-b6df-7a51f511bf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.485130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd971b4-6ee5-4098-b6df-7a51f511bf9f admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.485872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd971b4-6ee5-4098-b6df-7a51f511bf9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:14.488475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd971b4-6ee5-4098-b6df-7a51f511bf9f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:14.499501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 992/1986] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:14.504302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:14.517529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089aabd8-c945-413c-93cf-0d21e17887fb admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:14.522323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0a57c-87dc-4471-9e52-65620f83b457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:14.523197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0a57c-87dc-4471-9e52-65620f83b457 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.523521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0a57c-87dc-4471-9e52-65620f83b457 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.523796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0a57c-87dc-4471-9e52-65620f83b457 admin admin] PATH_INFO: `/v3/projects/99974dd0d2e446128ce49fff65ce91d7/users/a0824c643a0048ddb84f7ad3387a1eca/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.539435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0a57c-87dc-4471-9e52-65620f83b457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99974dd0d2e446128ce49fff65ce91d7, user_id=a0824c643a0048ddb84f7ad3387a1eca, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:14.542417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0a57c-87dc-4471-9e52-65620f83b457 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:14.588510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 995/1987] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:14 2022] PUT /identity/v3/projects/99974dd0d2e446128ce49fff65ce91d7/users/a0824c643a0048ddb84f7ad3387a1eca/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:14.599708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2235f3-4476-4fca-8dcc-d740b61ebc31 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.600175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2235f3-4476-4fca-8dcc-d740b61ebc31 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.600594 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2235f3-4476-4fca-8dcc-d740b61ebc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.629700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-cb2235f3-4476-4fca-8dcc-d740b61ebc31 None None] MFA Rules not processed for user `a0824c643a0048ddb84f7ad3387a1eca`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:14.631500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-cb2235f3-4476-4fca-8dcc-d740b61ebc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:14.698016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 993/1988] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:14.709468 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0a57c-87dc-4471-9e52-65620f83b457 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:14.717272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-bed0a57c-87dc-4471-9e52-65620f83b457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:14.718337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2235f3-4476-4fca-8dcc-d740b61ebc31 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:14.719744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-cb2235f3-4476-4fca-8dcc-d740b61ebc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:14.786914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0a57c-87dc-4471-9e52-65620f83b457 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:14.791318 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48724d40-7f12-4392-84da-9cc6d20b93d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:14.792018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48724d40-7f12-4392-84da-9cc6d20b93d2 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.792304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48724d40-7f12-4392-84da-9cc6d20b93d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.792578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48724d40-7f12-4392-84da-9cc6d20b93d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.792672 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2235f3-4476-4fca-8dcc-d740b61ebc31 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:14.798517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:14.799839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.800123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.800399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] PATH_INFO: `/v3/projects/2ba82caedf2b487e8bd25d3fed96c432` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.801600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba82caedf2b487e8bd25d3fed96c432)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:14.802117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48724d40-7f12-4392-84da-9cc6d20b93d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:14.803572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:14.803801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48724d40-7f12-4392-84da-9cc6d20b93d2 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:14.812815 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 996/1989] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:14.921765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba82caedf2b487e8bd25d3fed96c432', 'request_id': 'req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:14.928808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba82caedf2b487e8bd25d3fed96c432', 'request_id': 'req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:14.934580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba82caedf2b487e8bd25d3fed96c432', 'request_id': 'req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.010551 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] The token cache is being invalidate because project 2ba82caedf2b487e8bd25d3fed96c432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:15.010849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.011507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.013818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 994/1990] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:14 2022] DELETE /identity/v3/projects/2ba82caedf2b487e8bd25d3fed96c432 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:15.018527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48724d40-7f12-4392-84da-9cc6d20b93d2 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.029982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48724d40-7f12-4392-84da-9cc6d20b93d2 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:15.094310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:15.095402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.096154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.096430 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] PATH_INFO: `/v3/users/290d077cfbeb4fc8b33687b360e48b08` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.097628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] RBAC: Authorizing `identity:delete_user(user_id=290d077cfbeb4fc8b33687b360e48b08)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:15.099619 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:15.356463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:15.369786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '290d077cfbeb4fc8b33687b360e48b08'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.377367 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '290d077cfbeb4fc8b33687b360e48b08'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.474791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '290d077cfbeb4fc8b33687b360e48b08'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.474915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:15.475009 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '290d077cfbeb4fc8b33687b360e48b08'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.475009 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-487d4a80-7d07-43cc-a0a0-080ebbc4bbba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '290d077cfbeb4fc8b33687b360e48b08'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.490163 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 997/1991] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:15 2022] DELETE /identity/v3/users/290d077cfbeb4fc8b33687b360e48b08 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:15.790575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.792109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:15.819492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600d97e-e7c6-4494-8ae7-c71fd255ba1c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:15.887110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:15.888019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.888305 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.888584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] PATH_INFO: `/v3/projects/4a5c8c262af448bea543d424b5980ec6` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.889882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5c8c262af448bea543d424b5980ec6)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:15.891906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:16.041232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5c8c262af448bea543d424b5980ec6', 'request_id': 'req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:16.047800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5c8c262af448bea543d424b5980ec6', 'request_id': 'req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:16.054459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5c8c262af448bea543d424b5980ec6', 'request_id': 'req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:16.165383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] The token cache is being invalidate because project 4a5c8c262af448bea543d424b5980ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:16.165669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:16.166326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-93fc2bc9-31fd-4fc3-87c6-980719b1d31b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:16.167900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 995/1992] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:15 2022] DELETE /identity/v3/projects/4a5c8c262af448bea543d424b5980ec6 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:16.173234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4b074-22f1-4a5e-9a3c-4db315d3541e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.173690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4b074-22f1-4a5e-9a3c-4db315d3541e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.173781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4b074-22f1-4a5e-9a3c-4db315d3541e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.192741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f5a4b074-22f1-4a5e-9a3c-4db315d3541e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:16.193950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f5a4b074-22f1-4a5e-9a3c-4db315d3541e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:16.252665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 998/1993] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:16.260721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a18f4-59b4-473b-88cd-e333abb8dbab None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.261005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a18f4-59b4-473b-88cd-e333abb8dbab None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.261279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a18f4-59b4-473b-88cd-e333abb8dbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.279173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-f32a18f4-59b4-473b-88cd-e333abb8dbab None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:16.280413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f32a18f4-59b4-473b-88cd-e333abb8dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:16.289001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 996/1994] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:16.293415 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4b074-22f1-4a5e-9a3c-4db315d3541e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:16.303750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4b074-22f1-4a5e-9a3c-4db315d3541e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:16.308369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eacf59-ea44-4d24-818d-6b906794b7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:16.309026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eacf59-ea44-4d24-818d-6b906794b7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.309311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eacf59-ea44-4d24-818d-6b906794b7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.309590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eacf59-ea44-4d24-818d-6b906794b7b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.310318 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eacf59-ea44-4d24-818d-6b906794b7b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:16.310666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eacf59-ea44-4d24-818d-6b906794b7b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:16.312340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eacf59-ea44-4d24-818d-6b906794b7b1 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:16.324364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 999/1995] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:16.328835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a18f4-59b4-473b-88cd-e333abb8dbab None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:16.340359 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a18f4-59b4-473b-88cd-e333abb8dbab None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:16.345617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6204d-dbcb-41a4-a012-639d656575a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:16.346337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6204d-dbcb-41a4-a012-639d656575a6 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.346593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6204d-dbcb-41a4-a012-639d656575a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.347000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6204d-dbcb-41a4-a012-639d656575a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.348298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6204d-dbcb-41a4-a012-639d656575a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:16.357863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6204d-dbcb-41a4-a012-639d656575a6 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:16.465267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 997/1996] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:16 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:16.467967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eacf59-ea44-4d24-818d-6b906794b7b1 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:16.478462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eacf59-ea44-4d24-818d-6b906794b7b1 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:16.534012 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd57dd-338c-42e2-876d-9ebf5e38f785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:16.535382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd57dd-338c-42e2-876d-9ebf5e38f785 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.535660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd57dd-338c-42e2-876d-9ebf5e38f785 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.537672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd57dd-338c-42e2-876d-9ebf5e38f785 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.537672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd57dd-338c-42e2-876d-9ebf5e38f785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:16.538977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd57dd-338c-42e2-876d-9ebf5e38f785 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:16.605893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1000/1997] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:16 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:16.610157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6204d-dbcb-41a4-a012-639d656575a6 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:16.621279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6204d-dbcb-41a4-a012-639d656575a6 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:16.626400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be272c77-93bd-4998-80d9-9506aab85842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:16.627290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be272c77-93bd-4998-80d9-9506aab85842 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.627569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be272c77-93bd-4998-80d9-9506aab85842 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.627969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be272c77-93bd-4998-80d9-9506aab85842 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.628695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be272c77-93bd-4998-80d9-9506aab85842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:16.630393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be272c77-93bd-4998-80d9-9506aab85842 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:16.641103 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 998/1998] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:16.645624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd57dd-338c-42e2-876d-9ebf5e38f785 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:16.664457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd57dd-338c-42e2-876d-9ebf5e38f785 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:16.670241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4236d-fa29-40eb-9cc9-57eec033b77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:16.671006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4236d-fa29-40eb-9cc9-57eec033b77c admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.671293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4236d-fa29-40eb-9cc9-57eec033b77c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.671768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4236d-fa29-40eb-9cc9-57eec033b77c admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.672538 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4236d-fa29-40eb-9cc9-57eec033b77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:16.674218 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4236d-fa29-40eb-9cc9-57eec033b77c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:16.687051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1001/1999] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:16.691219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be272c77-93bd-4998-80d9-9506aab85842 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:16.704454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be272c77-93bd-4998-80d9-9506aab85842 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:16.707091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbe236-8af2-4760-90ae-d31bff1acd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:16.708007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe236-8af2-4760-90ae-d31bff1acd5e admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.708298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe236-8af2-4760-90ae-d31bff1acd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.708635 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe236-8af2-4760-90ae-d31bff1acd5e admin admin] PATH_INFO: `/v3/projects/47fcd069b23f4e978d46ffdd4f91bf6d/users/f7020b51aa2646e0aae56874985d8695/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.725983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbe236-8af2-4760-90ae-d31bff1acd5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47fcd069b23f4e978d46ffdd4f91bf6d, user_id=f7020b51aa2646e0aae56874985d8695, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:16.728763 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbe236-8af2-4760-90ae-d31bff1acd5e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:16.823880 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:16.829661 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 999/2000] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:16 2022] PUT /identity/v3/projects/47fcd069b23f4e978d46ffdd4f91bf6d/users/f7020b51aa2646e0aae56874985d8695/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:16.848951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f27acc-7791-4d70-a9c8-bd0c8e2f8a10 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.849578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f27acc-7791-4d70-a9c8-bd0c8e2f8a10 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.849789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f27acc-7791-4d70-a9c8-bd0c8e2f8a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.881678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e6f27acc-7791-4d70-a9c8-bd0c8e2f8a10 None None] MFA Rules not processed for user `f7020b51aa2646e0aae56874985d8695`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:16.882971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e6f27acc-7791-4d70-a9c8-bd0c8e2f8a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:16.887102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:16.956191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1002/2001] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:16.960637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbe236-8af2-4760-90ae-d31bff1acd5e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.026003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbe236-8af2-4760-90ae-d31bff1acd5e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.030983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21a7fc-224b-4b0a-a021-e705d785666f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.031652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21a7fc-224b-4b0a-a021-e705d785666f admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.032030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21a7fc-224b-4b0a-a021-e705d785666f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.032310 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21a7fc-224b-4b0a-a021-e705d785666f admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.033051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21a7fc-224b-4b0a-a021-e705d785666f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.035024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21a7fc-224b-4b0a-a021-e705d785666f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.109534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1000/2002] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:16 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:17.113599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f27acc-7791-4d70-a9c8-bd0c8e2f8a10 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.124099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f27acc-7791-4d70-a9c8-bd0c8e2f8a10 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.180212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7089589-e8d4-4885-9a64-c420d8a37849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.180931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7089589-e8d4-4885-9a64-c420d8a37849 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.181210 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7089589-e8d4-4885-9a64-c420d8a37849 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.181481 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7089589-e8d4-4885-9a64-c420d8a37849 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.182163 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7089589-e8d4-4885-9a64-c420d8a37849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.184115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7089589-e8d4-4885-9a64-c420d8a37849 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.240428 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1003/2003] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:17 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:17.244317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21a7fc-224b-4b0a-a021-e705d785666f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.255157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21a7fc-224b-4b0a-a021-e705d785666f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.260072 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cbd2d-75be-4800-a2ff-25b01cc9e935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.260777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cbd2d-75be-4800-a2ff-25b01cc9e935 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.261061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cbd2d-75be-4800-a2ff-25b01cc9e935 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.261339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cbd2d-75be-4800-a2ff-25b01cc9e935 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.262063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cbd2d-75be-4800-a2ff-25b01cc9e935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.263682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cbd2d-75be-4800-a2ff-25b01cc9e935 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.274452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1001/2004] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:17.279368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7089589-e8d4-4885-9a64-c420d8a37849 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.289689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7089589-e8d4-4885-9a64-c420d8a37849 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.294823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab8d44-0d92-481d-ba52-19fe7d8b1a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.295711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab8d44-0d92-481d-ba52-19fe7d8b1a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.295992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab8d44-0d92-481d-ba52-19fe7d8b1a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.296264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab8d44-0d92-481d-ba52-19fe7d8b1a05 admin admin] PATH_INFO: `/v3/projects/11ac8994b0954ab1a2ffb6f00b6a04c0/users/7dae6548447a4cc3a0e03be454fa30f2/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.311508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab8d44-0d92-481d-ba52-19fe7d8b1a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11ac8994b0954ab1a2ffb6f00b6a04c0, user_id=7dae6548447a4cc3a0e03be454fa30f2, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.314217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab8d44-0d92-481d-ba52-19fe7d8b1a05 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.388948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1004/2005] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:17 2022] PUT /identity/v3/projects/11ac8994b0954ab1a2ffb6f00b6a04c0/users/7dae6548447a4cc3a0e03be454fa30f2/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:17.392947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cbd2d-75be-4800-a2ff-25b01cc9e935 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.403188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cbd2d-75be-4800-a2ff-25b01cc9e935 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.465465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b3d5e-c1d3-4c9f-9061-e8b756d897dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.466340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b3d5e-c1d3-4c9f-9061-e8b756d897dc admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.466762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b3d5e-c1d3-4c9f-9061-e8b756d897dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.467156 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b3d5e-c1d3-4c9f-9061-e8b756d897dc admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.468023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b3d5e-c1d3-4c9f-9061-e8b756d897dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.469618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b3d5e-c1d3-4c9f-9061-e8b756d897dc admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.483197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1002/2006] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:17.484303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab8d44-0d92-481d-ba52-19fe7d8b1a05 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.495229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab8d44-0d92-481d-ba52-19fe7d8b1a05 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.503944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6293a-00bc-4ec4-971b-66ec4410e71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.503944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6293a-00bc-4ec4-971b-66ec4410e71f admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.503944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6293a-00bc-4ec4-971b-66ec4410e71f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.503944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6293a-00bc-4ec4-971b-66ec4410e71f admin admin] PATH_INFO: `/v3/projects/11ac8994b0954ab1a2ffb6f00b6a04c0/users/7dae6548447a4cc3a0e03be454fa30f2/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.504557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6293a-00bc-4ec4-971b-66ec4410e71f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11ac8994b0954ab1a2ffb6f00b6a04c0, user_id=7dae6548447a4cc3a0e03be454fa30f2, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.507020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6293a-00bc-4ec4-971b-66ec4410e71f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.517549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1005/2007] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:17 2022] PUT /identity/v3/projects/11ac8994b0954ab1a2ffb6f00b6a04c0/users/7dae6548447a4cc3a0e03be454fa30f2/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:17.527730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191eb696-8508-4ca2-9fda-ee9db6ce2494 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.528017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191eb696-8508-4ca2-9fda-ee9db6ce2494 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.528297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191eb696-8508-4ca2-9fda-ee9db6ce2494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.555395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-191eb696-8508-4ca2-9fda-ee9db6ce2494 None None] MFA Rules not processed for user `7dae6548447a4cc3a0e03be454fa30f2`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:17.556636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-191eb696-8508-4ca2-9fda-ee9db6ce2494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:17.619818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:17.629033 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1003/2008] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:17.640294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6293a-00bc-4ec4-971b-66ec4410e71f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.641957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-22b6293a-00bc-4ec4-971b-66ec4410e71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:17.700390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6293a-00bc-4ec4-971b-66ec4410e71f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.704058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07f016-d54a-4317-a3f5-4e5966e12770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.704698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07f016-d54a-4317-a3f5-4e5966e12770 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.704977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07f016-d54a-4317-a3f5-4e5966e12770 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.705253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07f016-d54a-4317-a3f5-4e5966e12770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.714949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:17.761886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07f016-d54a-4317-a3f5-4e5966e12770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.763284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07f016-d54a-4317-a3f5-4e5966e12770 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.772224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1006/2009] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:20.091163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191eb696-8508-4ca2-9fda-ee9db6ce2494 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:20.092644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-191eb696-8508-4ca2-9fda-ee9db6ce2494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:20.156106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191eb696-8508-4ca2-9fda-ee9db6ce2494 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:20.161289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:20.162126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:20.162402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:20.162991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] PATH_INFO: `/v3/users/a0824c643a0048ddb84f7ad3387a1eca` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:20.164179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0824c643a0048ddb84f7ad3387a1eca)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:20.166102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:20.349978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0824c643a0048ddb84f7ad3387a1eca'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:20.356758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0824c643a0048ddb84f7ad3387a1eca'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:20.357413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0824c643a0048ddb84f7ad3387a1eca'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:20.410522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0824c643a0048ddb84f7ad3387a1eca'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:20.411141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0824c643a0048ddb84f7ad3387a1eca'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:20.428536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1004/2010] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:20 2022] DELETE /identity/v3/users/a0824c643a0048ddb84f7ad3387a1eca => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:20.432993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9672970-4e09-4057-bb78-7f14e04fdf00 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:20.433486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9672970-4e09-4057-bb78-7f14e04fdf00 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:20.433611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9672970-4e09-4057-bb78-7f14e04fdf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:20.451777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a9672970-4e09-4057-bb78-7f14e04fdf00 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:20.452981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a9672970-4e09-4057-bb78-7f14e04fdf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:20.521068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1007/2011] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:20.536027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:20.537899 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:20.619914 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d01f6a-3df2-43e0-ab0a-c7de6274cc2b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:20.624516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae62b41-51b8-4873-ad8c-607350ea43cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:20.625738 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae62b41-51b8-4873-ad8c-607350ea43cd service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:20.625903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae62b41-51b8-4873-ad8c-607350ea43cd service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:20.626355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae62b41-51b8-4873-ad8c-607350ea43cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:20.639105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae62b41-51b8-4873-ad8c-607350ea43cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:20.640678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae62b41-51b8-4873-ad8c-607350ea43cd service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:20.657327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1005/2012] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:21.092658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9672970-4e09-4057-bb78-7f14e04fdf00 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:21.098273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a9672970-4e09-4057-bb78-7f14e04fdf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:21.126823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9672970-4e09-4057-bb78-7f14e04fdf00 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:21.133251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:21.134751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.135359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.135921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] PATH_INFO: `/v3/projects/99974dd0d2e446128ce49fff65ce91d7` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.137663 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99974dd0d2e446128ce49fff65ce91d7)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:21.140564 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:21.262960 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99974dd0d2e446128ce49fff65ce91d7', 'request_id': 'req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.269714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99974dd0d2e446128ce49fff65ce91d7', 'request_id': 'req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.275736 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99974dd0d2e446128ce49fff65ce91d7', 'request_id': 'req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.342090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] The token cache is being invalidate because project 99974dd0d2e446128ce49fff65ce91d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:21.342375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.345417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a7a11d8c-3878-4301-ac56-fe8bc5a0e43f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.345417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1008/2013] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:21 2022] DELETE /identity/v3/projects/99974dd0d2e446128ce49fff65ce91d7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:21.350508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ead62-ec9d-4585-bc43-e8fce92d994f None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.350984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ead62-ec9d-4585-bc43-e8fce92d994f None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.351220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ead62-ec9d-4585-bc43-e8fce92d994f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.370210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-094ead62-ec9d-4585-bc43-e8fce92d994f None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:21.371654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-094ead62-ec9d-4585-bc43-e8fce92d994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:21.390038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:21.442390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1006/2014] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:21.450200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba7a9-6171-4b62-a1a1-97ba5500c359 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.450790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba7a9-6171-4b62-a1a1-97ba5500c359 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.450790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba7a9-6171-4b62-a1a1-97ba5500c359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.469362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1a8ba7a9-6171-4b62-a1a1-97ba5500c359 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:21.470616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1a8ba7a9-6171-4b62-a1a1-97ba5500c359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:21.482664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1009/2015] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:21.483803 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ead62-ec9d-4585-bc43-e8fce92d994f None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:21.495950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ead62-ec9d-4585-bc43-e8fce92d994f None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:21.502649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213b3a1-986e-483e-9467-8876c508a3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:21.503340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213b3a1-986e-483e-9467-8876c508a3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.503657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213b3a1-986e-483e-9467-8876c508a3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.504088 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213b3a1-986e-483e-9467-8876c508a3a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.506091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213b3a1-986e-483e-9467-8876c508a3a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:21.506409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213b3a1-986e-483e-9467-8876c508a3a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:21.511765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213b3a1-986e-483e-9467-8876c508a3a4 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:21.524370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1007/2016] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:21.525475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba7a9-6171-4b62-a1a1-97ba5500c359 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:21.534875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba7a9-6171-4b62-a1a1-97ba5500c359 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:21.540738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461b185-c14c-45b2-849a-5752b07cb818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:21.541424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461b185-c14c-45b2-849a-5752b07cb818 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.541704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461b185-c14c-45b2-849a-5752b07cb818 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.541980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461b185-c14c-45b2-849a-5752b07cb818 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.542720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461b185-c14c-45b2-849a-5752b07cb818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:21.544683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461b185-c14c-45b2-849a-5752b07cb818 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:21.634749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1010/2017] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:21 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:21.638907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213b3a1-986e-483e-9467-8876c508a3a4 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:21.649383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213b3a1-986e-483e-9467-8876c508a3a4 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:21.705677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628a9e7-3254-4c9d-808a-7ec45fc6a393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:21.706309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628a9e7-3254-4c9d-808a-7ec45fc6a393 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.706618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628a9e7-3254-4c9d-808a-7ec45fc6a393 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.706896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628a9e7-3254-4c9d-808a-7ec45fc6a393 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.708225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628a9e7-3254-4c9d-808a-7ec45fc6a393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:21.710138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628a9e7-3254-4c9d-808a-7ec45fc6a393 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:21.840315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1008/2018] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:21 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:21.847952 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461b185-c14c-45b2-849a-5752b07cb818 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:21.858151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461b185-c14c-45b2-849a-5752b07cb818 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:21.862734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c058c-368e-4bb8-b7d7-2e37a7b701fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:21.863391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c058c-368e-4bb8-b7d7-2e37a7b701fd admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.863670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c058c-368e-4bb8-b7d7-2e37a7b701fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.863958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c058c-368e-4bb8-b7d7-2e37a7b701fd admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.864744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c058c-368e-4bb8-b7d7-2e37a7b701fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:21.866254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c058c-368e-4bb8-b7d7-2e37a7b701fd admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:21.867596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:21.879475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1011/2019] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:21.883380 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628a9e7-3254-4c9d-808a-7ec45fc6a393 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:21.899967 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628a9e7-3254-4c9d-808a-7ec45fc6a393 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:21.905049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2238d-cb16-4047-960a-d5575119d2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:21.905734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2238d-cb16-4047-960a-d5575119d2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.908579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2238d-cb16-4047-960a-d5575119d2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.908579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2238d-cb16-4047-960a-d5575119d2a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.908579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2238d-cb16-4047-960a-d5575119d2a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:21.908849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2238d-cb16-4047-960a-d5575119d2a1 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:21.919968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1009/2020] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:21.924154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c058c-368e-4bb8-b7d7-2e37a7b701fd admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:21.934485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c058c-368e-4bb8-b7d7-2e37a7b701fd admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:21.939195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d50c6d-a460-47cf-a2fe-ea1e20b0bffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:21.942817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d50c6d-a460-47cf-a2fe-ea1e20b0bffa admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.942817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d50c6d-a460-47cf-a2fe-ea1e20b0bffa admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.942817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d50c6d-a460-47cf-a2fe-ea1e20b0bffa admin admin] PATH_INFO: `/v3/projects/d9f37a52ba6146e5b1cccfdba0c60a18/users/99feeb6cf0fa46b4aabecefcdf56deff/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.958281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d50c6d-a460-47cf-a2fe-ea1e20b0bffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f37a52ba6146e5b1cccfdba0c60a18, user_id=99feeb6cf0fa46b4aabecefcdf56deff, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:21.961418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d50c6d-a460-47cf-a2fe-ea1e20b0bffa admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:22.069363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1012/2021] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:21 2022] PUT /identity/v3/projects/d9f37a52ba6146e5b1cccfdba0c60a18/users/99feeb6cf0fa46b4aabecefcdf56deff/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:22.074402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e9bc6-3ead-451b-8630-5f53993c6af9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:22.074723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e9bc6-3ead-451b-8630-5f53993c6af9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:22.075028 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e9bc6-3ead-451b-8630-5f53993c6af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:22.093046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-d83e9bc6-3ead-451b-8630-5f53993c6af9 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:22.094254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d83e9bc6-3ead-451b-8630-5f53993c6af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:22.155905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1010/2022] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:22.165694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d50c6d-a460-47cf-a2fe-ea1e20b0bffa admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:22.167447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-27d50c6d-a460-47cf-a2fe-ea1e20b0bffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:22.230436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d50c6d-a460-47cf-a2fe-ea1e20b0bffa admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:22.234160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff8f89-76b4-41ea-b9f6-8d82f5b52735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:22.234848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8f89-76b4-41ea-b9f6-8d82f5b52735 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:22.235135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8f89-76b4-41ea-b9f6-8d82f5b52735 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:22.235411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8f89-76b4-41ea-b9f6-8d82f5b52735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:22.245502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff8f89-76b4-41ea-b9f6-8d82f5b52735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:22.246979 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff8f89-76b4-41ea-b9f6-8d82f5b52735 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:22.261217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1013/2023] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:23.684330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57af7e3-c2af-49eb-9e77-1dcef7fa6697 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:23.685203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57af7e3-c2af-49eb-9e77-1dcef7fa6697 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:23.685464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57af7e3-c2af-49eb-9e77-1dcef7fa6697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:23.717162 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c57af7e3-c2af-49eb-9e77-1dcef7fa6697 None None] MFA Rules not processed for user `99feeb6cf0fa46b4aabecefcdf56deff`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:23.718602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c57af7e3-c2af-49eb-9e77-1dcef7fa6697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:23.738254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:23.807738 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1011/2024] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:23.819438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff8f89-76b4-41ea-b9f6-8d82f5b52735 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:23.820780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-05ff8f89-76b4-41ea-b9f6-8d82f5b52735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:23.881131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff8f89-76b4-41ea-b9f6-8d82f5b52735 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:23.884831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63252093-3595-4ff5-881f-834eb1a1e081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:23.885477 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63252093-3595-4ff5-881f-834eb1a1e081 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:23.885804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63252093-3595-4ff5-881f-834eb1a1e081 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:23.886063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63252093-3595-4ff5-881f-834eb1a1e081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:23.894744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63252093-3595-4ff5-881f-834eb1a1e081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:23.896239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63252093-3595-4ff5-881f-834eb1a1e081 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:23.908788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1014/2025] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:24.109241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57af7e3-c2af-49eb-9e77-1dcef7fa6697 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:24.110717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c57af7e3-c2af-49eb-9e77-1dcef7fa6697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:24.122997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57af7e3-c2af-49eb-9e77-1dcef7fa6697 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:24.126684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3487b6d-d816-4994-991e-0b3085b76257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:24.127329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3487b6d-d816-4994-991e-0b3085b76257 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:24.127609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3487b6d-d816-4994-991e-0b3085b76257 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:24.127964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3487b6d-d816-4994-991e-0b3085b76257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:24.129821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e3487b6d-d816-4994-991e-0b3085b76257 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:24.195137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3487b6d-d816-4994-991e-0b3085b76257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:24.196578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3487b6d-d816-4994-991e-0b3085b76257 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:24.206487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1012/2026] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:25.219170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63252093-3595-4ff5-881f-834eb1a1e081 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.220791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-63252093-3595-4ff5-881f-834eb1a1e081 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:25.238749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63252093-3595-4ff5-881f-834eb1a1e081 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.245299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.246199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.246771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.247262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] PATH_INFO: `/v3/users/80be150f20ae4a6fb4d365059d336103` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.248558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80be150f20ae4a6fb4d365059d336103)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.251700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.453114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80be150f20ae4a6fb4d365059d336103'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:25.460309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80be150f20ae4a6fb4d365059d336103'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:25.537308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80be150f20ae4a6fb4d365059d336103'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:25.538245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80be150f20ae4a6fb4d365059d336103'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:25.538883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80be150f20ae4a6fb4d365059d336103'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:25.546902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:25.558576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1015/2027] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:25 2022] DELETE /identity/v3/users/80be150f20ae4a6fb4d365059d336103 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:25.563323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58db6a7-e244-45af-aa58-ce0b0c4bdde0 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.563897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58db6a7-e244-45af-aa58-ce0b0c4bdde0 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.563897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58db6a7-e244-45af-aa58-ce0b0c4bdde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.581935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e58db6a7-e244-45af-aa58-ce0b0c4bdde0 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:25.583212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e58db6a7-e244-45af-aa58-ce0b0c4bdde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:25.648076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1013/2028] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:25.658186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.659762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:25.723268 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489894a4-867f-492f-a567-55f3ba3c6503 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.727566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae27db9-151c-4752-993d-f0e4c749dc19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.728200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae27db9-151c-4752-993d-f0e4c749dc19 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.728724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae27db9-151c-4752-993d-f0e4c749dc19 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.729010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae27db9-151c-4752-993d-f0e4c749dc19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.737164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd8931-6f77-4e2a-80bf-ee7df6bd611e None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.737454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd8931-6f77-4e2a-80bf-ee7df6bd611e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.737733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd8931-6f77-4e2a-80bf-ee7df6bd611e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.740301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae27db9-151c-4752-993d-f0e4c749dc19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.742065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae27db9-151c-4752-993d-f0e4c749dc19 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.752974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1016/2029] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:25.765175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c5cd8931-6f77-4e2a-80bf-ee7df6bd611e None None] MFA Rules not processed for user `99feeb6cf0fa46b4aabecefcdf56deff`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:25.766413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c5cd8931-6f77-4e2a-80bf-ee7df6bd611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:25.821273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:25.833136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1014/2030] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:26.235453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae27db9-151c-4752-993d-f0e4c749dc19 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.238174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2ae27db9-151c-4752-993d-f0e4c749dc19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:26.264877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae27db9-151c-4752-993d-f0e4c749dc19 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:26.269539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:26.270955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.271454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.271970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] PATH_INFO: `/v3/projects/2bb6418e0f1441599929c77141079855` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.273291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb6418e0f1441599929c77141079855)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:26.275501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:26.442098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb6418e0f1441599929c77141079855', 'request_id': 'req-05840c40-cabf-4744-a55a-44058f9faed7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:26.456058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb6418e0f1441599929c77141079855', 'request_id': 'req-05840c40-cabf-4744-a55a-44058f9faed7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:26.464070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb6418e0f1441599929c77141079855', 'request_id': 'req-05840c40-cabf-4744-a55a-44058f9faed7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:26.635886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] The token cache is being invalidate because project 2bb6418e0f1441599929c77141079855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:26.636301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:26.636896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-05840c40-cabf-4744-a55a-44058f9faed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:26.638542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1017/2031] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:26 2022] DELETE /identity/v3/projects/2bb6418e0f1441599929c77141079855 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:26.643612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d78e81-2e77-4432-8601-0a94a6e1871c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.644070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d78e81-2e77-4432-8601-0a94a6e1871c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.644070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d78e81-2e77-4432-8601-0a94a6e1871c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.663147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c2d78e81-2e77-4432-8601-0a94a6e1871c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:26.664287 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c2d78e81-2e77-4432-8601-0a94a6e1871c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:26.743124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1015/2032] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:26.751454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867fc2c-3544-45ab-8077-28676af453fa None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.751740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867fc2c-3544-45ab-8077-28676af453fa None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.752511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867fc2c-3544-45ab-8077-28676af453fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.797951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-b867fc2c-3544-45ab-8077-28676af453fa None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:26.802015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b867fc2c-3544-45ab-8077-28676af453fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:26.816780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1018/2033] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:26.824023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d78e81-2e77-4432-8601-0a94a6e1871c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.851891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d78e81-2e77-4432-8601-0a94a6e1871c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:26.851891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2a514-0de4-4ee3-9783-55335cff1199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:26.851891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2a514-0de4-4ee3-9783-55335cff1199 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.851891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2a514-0de4-4ee3-9783-55335cff1199 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.852515 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2a514-0de4-4ee3-9783-55335cff1199 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.852838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2a514-0de4-4ee3-9783-55335cff1199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:26.853149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2a514-0de4-4ee3-9783-55335cff1199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:26.854793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2a514-0de4-4ee3-9783-55335cff1199 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:26.868436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1016/2034] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:26.873679 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867fc2c-3544-45ab-8077-28676af453fa None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.883564 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867fc2c-3544-45ab-8077-28676af453fa None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:26.888629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb31a8-1926-4894-b908-5fe6b7b7e6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:26.889291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb31a8-1926-4894-b908-5fe6b7b7e6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.889571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb31a8-1926-4894-b908-5fe6b7b7e6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.889848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb31a8-1926-4894-b908-5fe6b7b7e6f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.890607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb31a8-1926-4894-b908-5fe6b7b7e6f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:26.892514 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb31a8-1926-4894-b908-5fe6b7b7e6f2 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:26.960907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1019/2035] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:26 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:26.966711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2a514-0de4-4ee3-9783-55335cff1199 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.978859 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2a514-0de4-4ee3-9783-55335cff1199 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:27.037187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e1138-971a-463c-a622-71d1421c3699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:27.038152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e1138-971a-463c-a622-71d1421c3699 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.038431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e1138-971a-463c-a622-71d1421c3699 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.038741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e1138-971a-463c-a622-71d1421c3699 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.039597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e1138-971a-463c-a622-71d1421c3699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:27.041682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e1138-971a-463c-a622-71d1421c3699 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:27.171128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1017/2036] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:26 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:27.175688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb31a8-1926-4894-b908-5fe6b7b7e6f2 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:27.189274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb31a8-1926-4894-b908-5fe6b7b7e6f2 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:27.194716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cb4c5-5214-421f-b5cd-c1185742b340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:27.195430 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cb4c5-5214-421f-b5cd-c1185742b340 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.195709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cb4c5-5214-421f-b5cd-c1185742b340 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.195980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cb4c5-5214-421f-b5cd-c1185742b340 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.196700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1cb4c5-5214-421f-b5cd-c1185742b340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:27.198396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1cb4c5-5214-421f-b5cd-c1185742b340 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:27.209563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1020/2037] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:27.214035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e1138-971a-463c-a622-71d1421c3699 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:27.224000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e1138-971a-463c-a622-71d1421c3699 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:27.228678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63fb3f-c779-4a58-8e84-3c63db455455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:27.231832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63fb3f-c779-4a58-8e84-3c63db455455 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.231832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63fb3f-c779-4a58-8e84-3c63db455455 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.231832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63fb3f-c779-4a58-8e84-3c63db455455 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.231832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63fb3f-c779-4a58-8e84-3c63db455455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:27.232196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63fb3f-c779-4a58-8e84-3c63db455455 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:27.242296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1018/2038] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:27.246350 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cb4c5-5214-421f-b5cd-c1185742b340 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:27.257029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cb4c5-5214-421f-b5cd-c1185742b340 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:27.261521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d686e-6b02-44b3-8a4a-dce6967fb68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:27.262437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d686e-6b02-44b3-8a4a-dce6967fb68d admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.262743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d686e-6b02-44b3-8a4a-dce6967fb68d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.263017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d686e-6b02-44b3-8a4a-dce6967fb68d admin admin] PATH_INFO: `/v3/projects/011cdb0f00b9444c812ac101bc5b43e3/users/a01061da6fc0404facce47e669ec80af/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.278835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d686e-6b02-44b3-8a4a-dce6967fb68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=011cdb0f00b9444c812ac101bc5b43e3, user_id=a01061da6fc0404facce47e669ec80af, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:27.281644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d686e-6b02-44b3-8a4a-dce6967fb68d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:27.343206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1021/2039] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:27 2022] PUT /identity/v3/projects/011cdb0f00b9444c812ac101bc5b43e3/users/a01061da6fc0404facce47e669ec80af/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:27.348350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161b4e0-8043-4778-9833-75a1ea50cf82 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.348639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161b4e0-8043-4778-9833-75a1ea50cf82 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.348933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161b4e0-8043-4778-9833-75a1ea50cf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.367122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-6161b4e0-8043-4778-9833-75a1ea50cf82 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:27.368321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6161b4e0-8043-4778-9833-75a1ea50cf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:27.418865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:27.428229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1019/2040] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:27.438271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d686e-6b02-44b3-8a4a-dce6967fb68d admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:27.440070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-441d686e-6b02-44b3-8a4a-dce6967fb68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:27.454590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:27.498255 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d686e-6b02-44b3-8a4a-dce6967fb68d admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:27.502286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98fa09-5278-4187-b8f5-23d49de9f4f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:27.502974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98fa09-5278-4187-b8f5-23d49de9f4f4 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.503258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98fa09-5278-4187-b8f5-23d49de9f4f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.503535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98fa09-5278-4187-b8f5-23d49de9f4f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.513734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98fa09-5278-4187-b8f5-23d49de9f4f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:27.515206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98fa09-5278-4187-b8f5-23d49de9f4f4 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:27.525556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1022/2041] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:28.889513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cec46-68ba-42ad-b22b-16c35f59b964 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:28.889513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cec46-68ba-42ad-b22b-16c35f59b964 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:28.889513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cec46-68ba-42ad-b22b-16c35f59b964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:28.923619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-6a0cec46-68ba-42ad-b22b-16c35f59b964 None None] MFA Rules not processed for user `a01061da6fc0404facce47e669ec80af`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:28.924824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6a0cec46-68ba-42ad-b22b-16c35f59b964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:29.009098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1020/2042] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:29.019689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98fa09-5278-4187-b8f5-23d49de9f4f4 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:29.021183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6b98fa09-5278-4187-b8f5-23d49de9f4f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:29.101851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98fa09-5278-4187-b8f5-23d49de9f4f4 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:29.105658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48586d-8bbd-42e1-ad1f-b7f37f94a576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:29.106325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48586d-8bbd-42e1-ad1f-b7f37f94a576 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:29.106654 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48586d-8bbd-42e1-ad1f-b7f37f94a576 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:29.106928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48586d-8bbd-42e1-ad1f-b7f37f94a576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:29.116927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48586d-8bbd-42e1-ad1f-b7f37f94a576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:29.118623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48586d-8bbd-42e1-ad1f-b7f37f94a576 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:29.127915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1023/2043] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:29.660282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cec46-68ba-42ad-b22b-16c35f59b964 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:29.661848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6a0cec46-68ba-42ad-b22b-16c35f59b964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:29.700952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cec46-68ba-42ad-b22b-16c35f59b964 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:29.705575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:29.706464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:29.706847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:29.707255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] PATH_INFO: `/v3/users/84379615d1a54acd8147960c57a391dc` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:29.708847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84379615d1a54acd8147960c57a391dc)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:29.710847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:29.951740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84379615d1a54acd8147960c57a391dc'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:29.959497 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84379615d1a54acd8147960c57a391dc'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:29.960204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84379615d1a54acd8147960c57a391dc'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.057303 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84379615d1a54acd8147960c57a391dc'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.058075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84379615d1a54acd8147960c57a391dc'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.077538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1021/2044] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:29 2022] DELETE /identity/v3/users/84379615d1a54acd8147960c57a391dc => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:30.082024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbd1fa-bc5d-44a6-a594-2a63c687684e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:30.082513 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbd1fa-bc5d-44a6-a594-2a63c687684e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:30.082608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbd1fa-bc5d-44a6-a594-2a63c687684e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:30.105869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-bedbd1fa-bc5d-44a6-a594-2a63c687684e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:30.107168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-bedbd1fa-bc5d-44a6-a594-2a63c687684e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:30.178221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1024/2045] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:30.190637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:30.192049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:30.257164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8ca10-cf7a-4fee-abf3-91e6ba32a4c1 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:30.260935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c6dd9-bce3-468c-86e0-6b1916a06434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:30.262219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c6dd9-bce3-468c-86e0-6b1916a06434 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:30.262219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c6dd9-bce3-468c-86e0-6b1916a06434 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:30.262724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c6dd9-bce3-468c-86e0-6b1916a06434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:30.282720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c6dd9-bce3-468c-86e0-6b1916a06434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:30.284469 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c6dd9-bce3-468c-86e0-6b1916a06434 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:30.295347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1022/2046] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:30.572992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbd1fa-bc5d-44a6-a594-2a63c687684e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:30.574556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-bedbd1fa-bc5d-44a6-a594-2a63c687684e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:30.586871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbd1fa-bc5d-44a6-a594-2a63c687684e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:30.591535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:30.592510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:30.592884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:30.593161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] PATH_INFO: `/v3/users/f7020b51aa2646e0aae56874985d8695` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:30.594375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7020b51aa2646e0aae56874985d8695)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:30.596348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:30.792945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9162c1d-6484-4ab5-8f1a-8698a76298cf None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:30.793228 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9162c1d-6484-4ab5-8f1a-8698a76298cf None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:30.793844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9162c1d-6484-4ab5-8f1a-8698a76298cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:30.800565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:30.817021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a9162c1d-6484-4ab5-8f1a-8698a76298cf None None] MFA Rules not processed for user `a01061da6fc0404facce47e669ec80af`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:30.818822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a9162c1d-6484-4ab5-8f1a-8698a76298cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:30.823618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7020b51aa2646e0aae56874985d8695'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.828658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:30.832425 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7020b51aa2646e0aae56874985d8695'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.880108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1023/2047] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:30.881080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9162c1d-6484-4ab5-8f1a-8698a76298cf None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:30.882354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a9162c1d-6484-4ab5-8f1a-8698a76298cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:30.896498 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9162c1d-6484-4ab5-8f1a-8698a76298cf None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:30.901215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:30.902095 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:30.902381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:30.902693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] PATH_INFO: `/v3/projects/4bb8a8a686844154baecdf57a634c395` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:30.909727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bb8a8a686844154baecdf57a634c395)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:30.913068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:30.976324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7020b51aa2646e0aae56874985d8695'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.976886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7020b51aa2646e0aae56874985d8695'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.977497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a87f6dfc-8586-4a03-a7be-68c6bcb0e12d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7020b51aa2646e0aae56874985d8695'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.995644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1025/2048] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:30 2022] DELETE /identity/v3/users/f7020b51aa2646e0aae56874985d8695 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:30.999918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621ba25-7ab4-4a8f-a81c-f56b682fcf25 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:31.000194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621ba25-7ab4-4a8f-a81c-f56b682fcf25 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:31.000478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621ba25-7ab4-4a8f-a81c-f56b682fcf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:31.018074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-0621ba25-7ab4-4a8f-a81c-f56b682fcf25 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:31.019322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0621ba25-7ab4-4a8f-a81c-f56b682fcf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:31.058947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bb8a8a686844154baecdf57a634c395', 'request_id': 'req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.069299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bb8a8a686844154baecdf57a634c395', 'request_id': 'req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.077424 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bb8a8a686844154baecdf57a634c395', 'request_id': 'req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.116095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1026/2049] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:31.124211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621ba25-7ab4-4a8f-a81c-f56b682fcf25 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:31.125671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0621ba25-7ab4-4a8f-a81c-f56b682fcf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:31.159340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] The token cache is being invalidate because project 4bb8a8a686844154baecdf57a634c395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:31.159814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.160588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.166072 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1024/2050] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:30 2022] DELETE /identity/v3/projects/4bb8a8a686844154baecdf57a634c395 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:31.168628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:31.174131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:31.188115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621ba25-7ab4-4a8f-a81c-f56b682fcf25 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:31.244271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702681d-2ffb-4da8-a430-443feeca6aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:31.244271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702681d-2ffb-4da8-a430-443feeca6aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:31.244271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702681d-2ffb-4da8-a430-443feeca6aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:31.244271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702681d-2ffb-4da8-a430-443feeca6aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:31.246580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2702681d-2ffb-4da8-a430-443feeca6aa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:31.270815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ccff1-f1c4-4ce5-aff6-2116aae39079 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:31.277622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:31.279000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:31.279457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:31.279882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] PATH_INFO: `/v3/users/913fd2c237c64170b128b18a928d1caa` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:31.281447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=913fd2c237c64170b128b18a928d1caa)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:31.284512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:31.343621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702681d-2ffb-4da8-a430-443feeca6aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:31.345130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702681d-2ffb-4da8-a430-443feeca6aa8 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:31.357661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1027/2051] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:31.457236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:31.499778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913fd2c237c64170b128b18a928d1caa'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.507432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913fd2c237c64170b128b18a928d1caa'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.508145 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913fd2c237c64170b128b18a928d1caa'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.605048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913fd2c237c64170b128b18a928d1caa'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.605699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913fd2c237c64170b128b18a928d1caa'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.624516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1025/2052] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:31 2022] DELETE /identity/v3/users/913fd2c237c64170b128b18a928d1caa => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:31.714039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:32.081491 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702681d-2ffb-4da8-a430-443feeca6aa8 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.087306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2702681d-2ffb-4da8-a430-443feeca6aa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:32.099611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702681d-2ffb-4da8-a430-443feeca6aa8 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.157822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.158724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.159010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.159290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] PATH_INFO: `/v3/projects/47fcd069b23f4e978d46ffdd4f91bf6d` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.160484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47fcd069b23f4e978d46ffdd4f91bf6d)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.163024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.212438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.213935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:32.228220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2fa6b-8ef9-4d29-bc24-599dde036d6e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.233545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.234517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.234873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.235211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] PATH_INFO: `/v3/projects/62ef20196ac649f6971e3776af606d3b` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.236586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62ef20196ac649f6971e3776af606d3b)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.238833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.313964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47fcd069b23f4e978d46ffdd4f91bf6d', 'request_id': 'req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.320480 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47fcd069b23f4e978d46ffdd4f91bf6d', 'request_id': 'req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.325969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47fcd069b23f4e978d46ffdd4f91bf6d', 'request_id': 'req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.394771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62ef20196ac649f6971e3776af606d3b', 'request_id': 'req-0840e847-ca89-4054-82f5-10273f9bb1c1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.401471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62ef20196ac649f6971e3776af606d3b', 'request_id': 'req-0840e847-ca89-4054-82f5-10273f9bb1c1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.407024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62ef20196ac649f6971e3776af606d3b', 'request_id': 'req-0840e847-ca89-4054-82f5-10273f9bb1c1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.427653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] The token cache is being invalidate because project 47fcd069b23f4e978d46ffdd4f91bf6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:32.427960 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.428604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.430180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1028/2053] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:32 2022] DELETE /identity/v3/projects/47fcd069b23f4e978d46ffdd4f91bf6d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:32.434766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.435545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:32.503560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336af6f-3f8b-46df-b5bb-bca0c3e057e2 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.513506 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] The token cache is being invalidate because project 62ef20196ac649f6971e3776af606d3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:32.513795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.514499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0840e847-ca89-4054-82f5-10273f9bb1c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.516033 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1026/2054] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:32 2022] DELETE /identity/v3/projects/62ef20196ac649f6971e3776af606d3b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:32.520826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9e582-a830-4a93-8d4f-db8f5a9184b0 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.525212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9e582-a830-4a93-8d4f-db8f5a9184b0 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.525212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9e582-a830-4a93-8d4f-db8f5a9184b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.540381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-18c9e582-a830-4a93-8d4f-db8f5a9184b0 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:32.541644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-18c9e582-a830-4a93-8d4f-db8f5a9184b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:32.571325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.571325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.571325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.571325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] PATH_INFO: `/v3/users/7dae6548447a4cc3a0e03be454fa30f2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.571907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dae6548447a4cc3a0e03be454fa30f2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.573870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.593672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:32.607940 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1027/2055] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:32.616158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fe02e-5019-4bb8-af2a-519999cd72e9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.616475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fe02e-5019-4bb8-af2a-519999cd72e9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.616749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fe02e-5019-4bb8-af2a-519999cd72e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.635150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-901fe02e-5019-4bb8-af2a-519999cd72e9 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:32.636392 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-901fe02e-5019-4bb8-af2a-519999cd72e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:32.644309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1028/2056] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:32.648343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fe02e-5019-4bb8-af2a-519999cd72e9 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.657508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fe02e-5019-4bb8-af2a-519999cd72e9 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.662129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abf8d4-dfd0-4e78-9db8-fb467e1b3435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.663117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abf8d4-dfd0-4e78-9db8-fb467e1b3435 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.663400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abf8d4-dfd0-4e78-9db8-fb467e1b3435 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.663675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abf8d4-dfd0-4e78-9db8-fb467e1b3435 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.664396 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abf8d4-dfd0-4e78-9db8-fb467e1b3435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:32.664732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abf8d4-dfd0-4e78-9db8-fb467e1b3435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.666280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abf8d4-dfd0-4e78-9db8-fb467e1b3435 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.678735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1029/2057] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:32.682999 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abf8d4-dfd0-4e78-9db8-fb467e1b3435 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.694460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abf8d4-dfd0-4e78-9db8-fb467e1b3435 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.701151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b0e17-109b-4999-8385-693658a63c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.701800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b0e17-109b-4999-8385-693658a63c9b admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.702087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b0e17-109b-4999-8385-693658a63c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.702360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b0e17-109b-4999-8385-693658a63c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.703115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b0e17-109b-4999-8385-693658a63c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.705046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b0e17-109b-4999-8385-693658a63c9b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.849049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1030/2058] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:32 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:32.853164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b0e17-109b-4999-8385-693658a63c9b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.854664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dae6548447a4cc3a0e03be454fa30f2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.863619 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dae6548447a4cc3a0e03be454fa30f2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.920865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b0e17-109b-4999-8385-693658a63c9b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.925469 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab2279-2aa8-403a-9ce0-ef1e03587e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.926189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab2279-2aa8-403a-9ce0-ef1e03587e71 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.926493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab2279-2aa8-403a-9ce0-ef1e03587e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.926801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab2279-2aa8-403a-9ce0-ef1e03587e71 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.927481 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab2279-2aa8-403a-9ce0-ef1e03587e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.929885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab2279-2aa8-403a-9ce0-ef1e03587e71 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.952745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:33.009925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dae6548447a4cc3a0e03be454fa30f2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:33.010660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dae6548447a4cc3a0e03be454fa30f2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:33.011274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dae6548447a4cc3a0e03be454fa30f2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:33.026687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1031/2059] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:32 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:33.030852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab2279-2aa8-403a-9ce0-ef1e03587e71 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.034299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5cab2279-2aa8-403a-9ce0-ef1e03587e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:33.038845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1029/2060] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:32 2022] DELETE /identity/v3/users/7dae6548447a4cc3a0e03be454fa30f2 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:33.047596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab2279-2aa8-403a-9ce0-ef1e03587e71 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.117545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a042f-3b12-4598-8d35-0b0a6e8743fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.118613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a042f-3b12-4598-8d35-0b0a6e8743fb admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.119079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a042f-3b12-4598-8d35-0b0a6e8743fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.119933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a042f-3b12-4598-8d35-0b0a6e8743fb admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.120991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a042f-3b12-4598-8d35-0b0a6e8743fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.123532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a042f-3b12-4598-8d35-0b0a6e8743fb admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.136485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1032/2061] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:33.140125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.151738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb38f3-6af0-40fc-a32c-a358cd6eddff admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.156807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e875435-693a-4df2-bf42-211afc6314dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.158616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e875435-693a-4df2-bf42-211afc6314dd admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.158616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e875435-693a-4df2-bf42-211afc6314dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.158616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e875435-693a-4df2-bf42-211afc6314dd admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.159360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e875435-693a-4df2-bf42-211afc6314dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.160861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e875435-693a-4df2-bf42-211afc6314dd admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.172107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1030/2062] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:33.176521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a042f-3b12-4598-8d35-0b0a6e8743fb admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.187103 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a042f-3b12-4598-8d35-0b0a6e8743fb admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.191628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0caf07-f851-45b4-80e3-78f137d13d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.192510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0caf07-f851-45b4-80e3-78f137d13d5c admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.192794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0caf07-f851-45b4-80e3-78f137d13d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.193066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0caf07-f851-45b4-80e3-78f137d13d5c admin admin] PATH_INFO: `/v3/projects/3cb5eeaf783643c39e3da82beaccc119/users/04eb90a102284726a13e8b396eb9ac95/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.207745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0caf07-f851-45b4-80e3-78f137d13d5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb5eeaf783643c39e3da82beaccc119, user_id=04eb90a102284726a13e8b396eb9ac95, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.210417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0caf07-f851-45b4-80e3-78f137d13d5c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.265873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1033/2063] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:33 2022] PUT /identity/v3/projects/3cb5eeaf783643c39e3da82beaccc119/users/04eb90a102284726a13e8b396eb9ac95/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:33.276601 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dbaf2-dc76-4043-8094-fcb9772ac1b9 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.276891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dbaf2-dc76-4043-8094-fcb9772ac1b9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.277199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dbaf2-dc76-4043-8094-fcb9772ac1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.310803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-8f2dbaf2-dc76-4043-8094-fcb9772ac1b9 None None] MFA Rules not processed for user `04eb90a102284726a13e8b396eb9ac95`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:33.312086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8f2dbaf2-dc76-4043-8094-fcb9772ac1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:33.405311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1031/2064] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:33.409879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0caf07-f851-45b4-80e3-78f137d13d5c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.489293 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0caf07-f851-45b4-80e3-78f137d13d5c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.507013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab4552-75f6-4685-aec0-2796d5959cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.509102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4552-75f6-4685-aec0-2796d5959cad admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.515554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4552-75f6-4685-aec0-2796d5959cad admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.515554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4552-75f6-4685-aec0-2796d5959cad admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.515554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab4552-75f6-4685-aec0-2796d5959cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.515554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab4552-75f6-4685-aec0-2796d5959cad admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.521513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:33.613155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1034/2065] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:33 2022] POST /identity/v3/projects => generated 369 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:33.617631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dbaf2-dc76-4043-8094-fcb9772ac1b9 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.636372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dbaf2-dc76-4043-8094-fcb9772ac1b9 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.650363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:33.713320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15db95-b58a-43f3-b6a8-4eb3d4e0a102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.714018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15db95-b58a-43f3-b6a8-4eb3d4e0a102 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.714296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15db95-b58a-43f3-b6a8-4eb3d4e0a102 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.715182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15db95-b58a-43f3-b6a8-4eb3d4e0a102 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.715887 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15db95-b58a-43f3-b6a8-4eb3d4e0a102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.717833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15db95-b58a-43f3-b6a8-4eb3d4e0a102 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.806144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1032/2066] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:33 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:33.809841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab4552-75f6-4685-aec0-2796d5959cad admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.820920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab4552-75f6-4685-aec0-2796d5959cad admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.827064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531f5f1-16e8-4b61-8932-43e5759ebf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.827855 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531f5f1-16e8-4b61-8932-43e5759ebf9c admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.828147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531f5f1-16e8-4b61-8932-43e5759ebf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.828424 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531f5f1-16e8-4b61-8932-43e5759ebf9c admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.829245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531f5f1-16e8-4b61-8932-43e5759ebf9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.830117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15db95-b58a-43f3-b6a8-4eb3d4e0a102 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.830895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531f5f1-16e8-4b61-8932-43e5759ebf9c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.831995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9b15db95-b58a-43f3-b6a8-4eb3d4e0a102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:33.847953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1035/2067] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:33.849032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531f5f1-16e8-4b61-8932-43e5759ebf9c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.850160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15db95-b58a-43f3-b6a8-4eb3d4e0a102 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.873202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.873202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.873202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.873202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] PATH_INFO: `/v3/projects/11ac8994b0954ab1a2ffb6f00b6a04c0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.873931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=11ac8994b0954ab1a2ffb6f00b6a04c0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.874377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531f5f1-16e8-4b61-8932-43e5759ebf9c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.874737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.877739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708bcfc-dd17-4138-945c-e8e6b8ecda5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.878731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708bcfc-dd17-4138-945c-e8e6b8ecda5f admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.879084 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708bcfc-dd17-4138-945c-e8e6b8ecda5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.879430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708bcfc-dd17-4138-945c-e8e6b8ecda5f admin admin] PATH_INFO: `/v3/projects/b663dad87d35442b86ab77479abcf862/users/fabedb69fe3d428ca16a1d2244ad9481/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.895154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708bcfc-dd17-4138-945c-e8e6b8ecda5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b663dad87d35442b86ab77479abcf862, user_id=fabedb69fe3d428ca16a1d2244ad9481, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.898410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708bcfc-dd17-4138-945c-e8e6b8ecda5f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.984535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1036/2068] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:33 2022] PUT /identity/v3/projects/b663dad87d35442b86ab77479abcf862/users/fabedb69fe3d428ca16a1d2244ad9481/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:33.988683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708bcfc-dd17-4138-945c-e8e6b8ecda5f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.025843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11ac8994b0954ab1a2ffb6f00b6a04c0', 'request_id': 'req-fdc969f8-8921-400e-b4d3-1356921c27eb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:34.033250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11ac8994b0954ab1a2ffb6f00b6a04c0', 'request_id': 'req-fdc969f8-8921-400e-b4d3-1356921c27eb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:34.042832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11ac8994b0954ab1a2ffb6f00b6a04c0', 'request_id': 'req-fdc969f8-8921-400e-b4d3-1356921c27eb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:34.064812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708bcfc-dd17-4138-945c-e8e6b8ecda5f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.125689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ad3f3-218c-4830-8371-cf10f5b091f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.126446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ad3f3-218c-4830-8371-cf10f5b091f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.126793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ad3f3-218c-4830-8371-cf10f5b091f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.127073 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ad3f3-218c-4830-8371-cf10f5b091f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.127974 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ad3f3-218c-4830-8371-cf10f5b091f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.129596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ad3f3-218c-4830-8371-cf10f5b091f2 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.140736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1037/2069] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:34.144914 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ad3f3-218c-4830-8371-cf10f5b091f2 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.154666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] The token cache is being invalidate because project 11ac8994b0954ab1a2ffb6f00b6a04c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:34.155022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ad3f3-218c-4830-8371-cf10f5b091f2 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.155127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:34.155825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-fdc969f8-8921-400e-b4d3-1356921c27eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:34.157755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1033/2070] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:33 2022] DELETE /identity/v3/projects/11ac8994b0954ab1a2ffb6f00b6a04c0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:34.160448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7b4fb-56e5-4634-9786-7a014f130d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.161427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b4fb-56e5-4634-9786-7a014f130d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.161971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b4fb-56e5-4634-9786-7a014f130d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.162209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b4fb-56e5-4634-9786-7a014f130d80 admin admin] PATH_INFO: `/v3/projects/b663dad87d35442b86ab77479abcf862/users/fabedb69fe3d428ca16a1d2244ad9481/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.163739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1fe30-e84b-456c-a271-c4a10d63c205 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.164212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1fe30-e84b-456c-a271-c4a10d63c205 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.164363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1fe30-e84b-456c-a271-c4a10d63c205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.165254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7b4fb-56e5-4634-9786-7a014f130d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b663dad87d35442b86ab77479abcf862, user_id=fabedb69fe3d428ca16a1d2244ad9481, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.168089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7b4fb-56e5-4634-9786-7a014f130d80 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.187770 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1038/2071] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:34 2022] PUT /identity/v3/projects/b663dad87d35442b86ab77479abcf862/users/fabedb69fe3d428ca16a1d2244ad9481/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:34.187953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-70e1fe30-e84b-456c-a271-c4a10d63c205 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:34.187953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-70e1fe30-e84b-456c-a271-c4a10d63c205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.198772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28e42d-4f9f-45b3-8b7a-878af38f3440 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.199122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28e42d-4f9f-45b3-8b7a-878af38f3440 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.199417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28e42d-4f9f-45b3-8b7a-878af38f3440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.230048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ab28e42d-4f9f-45b3-8b7a-878af38f3440 None None] MFA Rules not processed for user `fabedb69fe3d428ca16a1d2244ad9481`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:34.231536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ab28e42d-4f9f-45b3-8b7a-878af38f3440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.261092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:34.265533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1034/2072] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:34.275244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306cf1a-0c02-402b-90be-d9eeb94875da None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.275794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306cf1a-0c02-402b-90be-d9eeb94875da None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.276074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306cf1a-0c02-402b-90be-d9eeb94875da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.295392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-b306cf1a-0c02-402b-90be-d9eeb94875da None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:34.296625 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b306cf1a-0c02-402b-90be-d9eeb94875da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.308894 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1035/2073] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:34.310827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306cf1a-0c02-402b-90be-d9eeb94875da None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.316285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1039/2074] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:34.322650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306cf1a-0c02-402b-90be-d9eeb94875da None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.327971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee459c9c-2a32-43d2-9e61-9d4c9010ec41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.328551 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28e42d-4f9f-45b3-8b7a-878af38f3440 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.330116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ab28e42d-4f9f-45b3-8b7a-878af38f3440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.335870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee459c9c-2a32-43d2-9e61-9d4c9010ec41 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.336417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee459c9c-2a32-43d2-9e61-9d4c9010ec41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.336829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee459c9c-2a32-43d2-9e61-9d4c9010ec41 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.338155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee459c9c-2a32-43d2-9e61-9d4c9010ec41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:34.338606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee459c9c-2a32-43d2-9e61-9d4c9010ec41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.341442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee459c9c-2a32-43d2-9e61-9d4c9010ec41 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.355307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1036/2075] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:34.359698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee459c9c-2a32-43d2-9e61-9d4c9010ec41 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.367932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:34.373086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee459c9c-2a32-43d2-9e61-9d4c9010ec41 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.379455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa456e-0c1d-448f-8e09-5fa846ee3a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.380127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa456e-0c1d-448f-8e09-5fa846ee3a41 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.380411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa456e-0c1d-448f-8e09-5fa846ee3a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.380688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa456e-0c1d-448f-8e09-5fa846ee3a41 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.381954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa456e-0c1d-448f-8e09-5fa846ee3a41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.383981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa456e-0c1d-448f-8e09-5fa846ee3a41 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.400565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28e42d-4f9f-45b3-8b7a-878af38f3440 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.404256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a500ede-b47c-4537-be65-ebba04fc6f8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.404905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a500ede-b47c-4537-be65-ebba04fc6f8d service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.405185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a500ede-b47c-4537-be65-ebba04fc6f8d service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.405459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a500ede-b47c-4537-be65-ebba04fc6f8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.414852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a500ede-b47c-4537-be65-ebba04fc6f8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.416730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a500ede-b47c-4537-be65-ebba04fc6f8d service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.426679 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1040/2076] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:34 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:34.491034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1037/2077] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/projects => generated 357 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:34.495472 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a500ede-b47c-4537-be65-ebba04fc6f8d service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.507604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a500ede-b47c-4537-be65-ebba04fc6f8d service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.561343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa456e-0c1d-448f-8e09-5fa846ee3a41 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.563742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-fcfa456e-0c1d-448f-8e09-5fa846ee3a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.578990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df74465-c69a-4480-b04c-be3565c99d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.579678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df74465-c69a-4480-b04c-be3565c99d7b admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.579964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df74465-c69a-4480-b04c-be3565c99d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.580243 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df74465-c69a-4480-b04c-be3565c99d7b admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.581169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df74465-c69a-4480-b04c-be3565c99d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.583323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df74465-c69a-4480-b04c-be3565c99d7b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.643284 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa456e-0c1d-448f-8e09-5fa846ee3a41 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.647206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922b2db-1992-4d5f-8868-e5f72701ffc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.647929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922b2db-1992-4d5f-8868-e5f72701ffc4 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.648218 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922b2db-1992-4d5f-8868-e5f72701ffc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.648494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922b2db-1992-4d5f-8868-e5f72701ffc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.650297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7922b2db-1992-4d5f-8868-e5f72701ffc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.694166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1041/2078] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/users => generated 279 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:34.697972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df74465-c69a-4480-b04c-be3565c99d7b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.719094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df74465-c69a-4480-b04c-be3565c99d7b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.731579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a0086-3502-48fd-8944-c4e03848f3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.734133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a0086-3502-48fd-8944-c4e03848f3ea admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.734631 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a0086-3502-48fd-8944-c4e03848f3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.734973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a0086-3502-48fd-8944-c4e03848f3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.736076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a0086-3502-48fd-8944-c4e03848f3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.738253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a0086-3502-48fd-8944-c4e03848f3ea admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.760578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1042/2079] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:34.764409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a0086-3502-48fd-8944-c4e03848f3ea admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.778028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922b2db-1992-4d5f-8868-e5f72701ffc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.779946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a0086-3502-48fd-8944-c4e03848f3ea admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.782797 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922b2db-1992-4d5f-8868-e5f72701ffc4 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.788389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60352728-39a6-4ece-a7df-3c5766ba3bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.789148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60352728-39a6-4ece-a7df-3c5766ba3bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.789603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60352728-39a6-4ece-a7df-3c5766ba3bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.800710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60352728-39a6-4ece-a7df-3c5766ba3bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.801581 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60352728-39a6-4ece-a7df-3c5766ba3bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.808477 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60352728-39a6-4ece-a7df-3c5766ba3bd6 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.816347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1038/2080] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:34.827326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1043/2081] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:34.832243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922b2db-1992-4d5f-8868-e5f72701ffc4 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.846132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922b2db-1992-4d5f-8868-e5f72701ffc4 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.852175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd694410-d452-41e2-973f-8989d21d7211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.853857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694410-d452-41e2-973f-8989d21d7211 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.854157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694410-d452-41e2-973f-8989d21d7211 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.854441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694410-d452-41e2-973f-8989d21d7211 admin admin] PATH_INFO: `/v3/projects/bf58d3e4a37845fdb71d841f71cd8451/users/4c314b102275446ebc070684905134d9/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.876762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd694410-d452-41e2-973f-8989d21d7211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf58d3e4a37845fdb71d841f71cd8451, user_id=4c314b102275446ebc070684905134d9, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.879570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd694410-d452-41e2-973f-8989d21d7211 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.983161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1039/2082] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:34 2022] PUT /identity/v3/projects/bf58d3e4a37845fdb71d841f71cd8451/users/4c314b102275446ebc070684905134d9/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:35.000498 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e18fa2-2bdc-4d5c-8683-71f060c33960 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:35.004719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e18fa2-2bdc-4d5c-8683-71f060c33960 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:35.005456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e18fa2-2bdc-4d5c-8683-71f060c33960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:35.047289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-b4e18fa2-2bdc-4d5c-8683-71f060c33960 None None] MFA Rules not processed for user `4c314b102275446ebc070684905134d9`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:35.048891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b4e18fa2-2bdc-4d5c-8683-71f060c33960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:35.087215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:35.164872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1044/2083] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:35.705552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd694410-d452-41e2-973f-8989d21d7211 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:35.764735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd694410-d452-41e2-973f-8989d21d7211 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:35.768406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7d959-7572-4013-a704-d249a209352a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:35.769061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d959-7572-4013-a704-d249a209352a service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:35.769347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d959-7572-4013-a704-d249a209352a service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:35.769644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d959-7572-4013-a704-d249a209352a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:35.778447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7d959-7572-4013-a704-d249a209352a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:35.779932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7d959-7572-4013-a704-d249a209352a service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:35.789264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1040/2084] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:35 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:35.805136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e18fa2-2bdc-4d5c-8683-71f060c33960 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:35.870860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e18fa2-2bdc-4d5c-8683-71f060c33960 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:35.875996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:35.876911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:35.877195 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:35.877490 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] PATH_INFO: `/v3/users/4c314b102275446ebc070684905134d9` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:35.878700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c314b102275446ebc070684905134d9)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:35.880615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:36.125632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c314b102275446ebc070684905134d9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.139040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c314b102275446ebc070684905134d9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.142736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c314b102275446ebc070684905134d9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.193579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c314b102275446ebc070684905134d9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.194448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c314b102275446ebc070684905134d9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.222760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1045/2085] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:35 2022] DELETE /identity/v3/users/4c314b102275446ebc070684905134d9 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:36.228078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb80fea-7396-4d9f-8c96-a4c06599f83b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:36.228201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb80fea-7396-4d9f-8c96-a4c06599f83b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:36.228649 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb80fea-7396-4d9f-8c96-a4c06599f83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:36.247226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-7fb80fea-7396-4d9f-8c96-a4c06599f83b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:36.248526 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7fb80fea-7396-4d9f-8c96-a4c06599f83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:36.257802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:36.324632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1041/2086] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:36.345500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:36.347263 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:36.403416 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:36.427146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d64500-50e5-4094-b586-ff0d90a1f3a3 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:36.433017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0ec57-492a-461c-b557-c3e7065a03e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:36.434308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0ec57-492a-461c-b557-c3e7065a03e1 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:36.434715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0ec57-492a-461c-b557-c3e7065a03e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:36.435108 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0ec57-492a-461c-b557-c3e7065a03e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:36.447917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0ec57-492a-461c-b557-c3e7065a03e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:36.450459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0ec57-492a-461c-b557-c3e7065a03e1 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:36.470393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1046/2087] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:37.082447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb80fea-7396-4d9f-8c96-a4c06599f83b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.084658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7fb80fea-7396-4d9f-8c96-a4c06599f83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:37.108032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb80fea-7396-4d9f-8c96-a4c06599f83b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.115407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.116308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.116598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.116881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] PATH_INFO: `/v3/projects/bf58d3e4a37845fdb71d841f71cd8451` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.119277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf58d3e4a37845fdb71d841f71cd8451)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.121520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.318793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf58d3e4a37845fdb71d841f71cd8451', 'request_id': 'req-d37af03a-40ef-4453-9972-a188992b35d5'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:37.327364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf58d3e4a37845fdb71d841f71cd8451', 'request_id': 'req-d37af03a-40ef-4453-9972-a188992b35d5'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:37.338703 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf58d3e4a37845fdb71d841f71cd8451', 'request_id': 'req-d37af03a-40ef-4453-9972-a188992b35d5'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:37.415719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] The token cache is being invalidate because project bf58d3e4a37845fdb71d841f71cd8451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:37.416014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:37.418940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d37af03a-40ef-4453-9972-a188992b35d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:37.421722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1042/2088] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:37 2022] DELETE /identity/v3/projects/bf58d3e4a37845fdb71d841f71cd8451 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:37.429504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aad15-cd18-490d-9eaf-0b54c80d56fc None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.429504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aad15-cd18-490d-9eaf-0b54c80d56fc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.429504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134aad15-cd18-490d-9eaf-0b54c80d56fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.447313 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-134aad15-cd18-490d-9eaf-0b54c80d56fc None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:37.448518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-134aad15-cd18-490d-9eaf-0b54c80d56fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:37.521701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1047/2089] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:37.530134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109e92f-867a-4700-bc55-35e493c0631b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.530420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109e92f-867a-4700-bc55-35e493c0631b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.530731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109e92f-867a-4700-bc55-35e493c0631b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.553641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f109e92f-867a-4700-bc55-35e493c0631b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:37.555013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f109e92f-867a-4700-bc55-35e493c0631b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:37.563181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1043/2090] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:37.567556 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aad15-cd18-490d-9eaf-0b54c80d56fc None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.577709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134aad15-cd18-490d-9eaf-0b54c80d56fc None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.582280 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff577b7b-2d15-46c4-943c-b91a57f85237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.582983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff577b7b-2d15-46c4-943c-b91a57f85237 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.583271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff577b7b-2d15-46c4-943c-b91a57f85237 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.583551 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff577b7b-2d15-46c4-943c-b91a57f85237 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.584285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff577b7b-2d15-46c4-943c-b91a57f85237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:37.584595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff577b7b-2d15-46c4-943c-b91a57f85237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.586199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff577b7b-2d15-46c4-943c-b91a57f85237 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.608966 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1048/2091] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:37.613291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109e92f-867a-4700-bc55-35e493c0631b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.629357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109e92f-867a-4700-bc55-35e493c0631b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.635845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f75ed0-4a0f-4104-af0f-84854f699945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.636509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f75ed0-4a0f-4104-af0f-84854f699945 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.636789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f75ed0-4a0f-4104-af0f-84854f699945 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.637556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f75ed0-4a0f-4104-af0f-84854f699945 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.637763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f75ed0-4a0f-4104-af0f-84854f699945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.640457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f75ed0-4a0f-4104-af0f-84854f699945 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.768427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1044/2092] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:37 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:37.773957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff577b7b-2d15-46c4-943c-b91a57f85237 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.784727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff577b7b-2d15-46c4-943c-b91a57f85237 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.881576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e36002-3c67-4a29-8d1d-f16ae9455255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.882262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e36002-3c67-4a29-8d1d-f16ae9455255 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.882609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e36002-3c67-4a29-8d1d-f16ae9455255 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.882969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e36002-3c67-4a29-8d1d-f16ae9455255 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.883665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e36002-3c67-4a29-8d1d-f16ae9455255 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.885616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e36002-3c67-4a29-8d1d-f16ae9455255 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.960377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1049/2093] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:37 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:37.964773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f75ed0-4a0f-4104-af0f-84854f699945 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.975686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f75ed0-4a0f-4104-af0f-84854f699945 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.980879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22516f-e131-4964-b259-fe02f0e5c91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.981580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22516f-e131-4964-b259-fe02f0e5c91c admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.981864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22516f-e131-4964-b259-fe02f0e5c91c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.982172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22516f-e131-4964-b259-fe02f0e5c91c admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.982969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22516f-e131-4964-b259-fe02f0e5c91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.984719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22516f-e131-4964-b259-fe02f0e5c91c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.996581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1045/2094] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:38.000577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e36002-3c67-4a29-8d1d-f16ae9455255 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:38.010800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e36002-3c67-4a29-8d1d-f16ae9455255 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:38.015298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606130a5-c650-404d-8275-7605f9295728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:38.015945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606130a5-c650-404d-8275-7605f9295728 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.016228 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606130a5-c650-404d-8275-7605f9295728 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.016501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606130a5-c650-404d-8275-7605f9295728 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.017215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606130a5-c650-404d-8275-7605f9295728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:38.018812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606130a5-c650-404d-8275-7605f9295728 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:38.029210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1050/2095] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:38.033272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22516f-e131-4964-b259-fe02f0e5c91c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:38.043954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22516f-e131-4964-b259-fe02f0e5c91c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:38.048576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c6930-4b38-4b64-bd60-cead6630706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:38.049455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6930-4b38-4b64-bd60-cead6630706c admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.049739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6930-4b38-4b64-bd60-cead6630706c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.050015 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6930-4b38-4b64-bd60-cead6630706c admin admin] PATH_INFO: `/v3/projects/52a1423a26e041959827c5d8611cffdf/users/d03b404df6d54b638804804bbc08e1cf/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.064397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c6930-4b38-4b64-bd60-cead6630706c admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a1423a26e041959827c5d8611cffdf, user_id=d03b404df6d54b638804804bbc08e1cf, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:38.067207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c6930-4b38-4b64-bd60-cead6630706c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:38.131526 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1046/2096] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:38 2022] PUT /identity/v3/projects/52a1423a26e041959827c5d8611cffdf/users/d03b404df6d54b638804804bbc08e1cf/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:38.138908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255a230-b981-4059-9e6a-47ff7be4b17a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.139193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255a230-b981-4059-9e6a-47ff7be4b17a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.139469 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255a230-b981-4059-9e6a-47ff7be4b17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.157904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0255a230-b981-4059-9e6a-47ff7be4b17a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:38.159193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0255a230-b981-4059-9e6a-47ff7be4b17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:38.203069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:38.219035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1051/2097] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:38.228453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c6930-4b38-4b64-bd60-cead6630706c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:38.233083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c32c6930-4b38-4b64-bd60-cead6630706c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:38.270939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:38.288346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c6930-4b38-4b64-bd60-cead6630706c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:38.292239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d73b0e-77fb-4cfa-b749-a1846e66b778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:38.292892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73b0e-77fb-4cfa-b749-a1846e66b778 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.293183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73b0e-77fb-4cfa-b749-a1846e66b778 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.293457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73b0e-77fb-4cfa-b749-a1846e66b778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.302985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d73b0e-77fb-4cfa-b749-a1846e66b778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:38.305155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d73b0e-77fb-4cfa-b749-a1846e66b778 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:38.314833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1047/2098] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:40.206886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255a230-b981-4059-9e6a-47ff7be4b17a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:40.210523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0255a230-b981-4059-9e6a-47ff7be4b17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:40.222868 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255a230-b981-4059-9e6a-47ff7be4b17a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:40.229076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:40.230086 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:40.230525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:40.230811 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] PATH_INFO: `/v3/users/04eb90a102284726a13e8b396eb9ac95` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:40.232220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04eb90a102284726a13e8b396eb9ac95)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:40.234224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:40.405484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04eb90a102284726a13e8b396eb9ac95'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:40.415464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04eb90a102284726a13e8b396eb9ac95'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:40.416721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04eb90a102284726a13e8b396eb9ac95'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:40.524819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04eb90a102284726a13e8b396eb9ac95'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:40.525821 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04eb90a102284726a13e8b396eb9ac95'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:40.560584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1052/2099] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:40 2022] DELETE /identity/v3/users/04eb90a102284726a13e8b396eb9ac95 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:40.562219 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f4639-4f2f-4c4b-a24f-d92f2fd63fa9 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:40.562735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f4639-4f2f-4c4b-a24f-d92f2fd63fa9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:40.610357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f4639-4f2f-4c4b-a24f-d92f2fd63fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:40.610357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-312f4639-4f2f-4c4b-a24f-d92f2fd63fa9 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:40.610357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-312f4639-4f2f-4c4b-a24f-d92f2fd63fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:40.717722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1048/2100] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:40.740067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:40.746740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:40.876347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07672052-c736-4128-be13-6630c78f4086 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:40.911963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245b409-a1a9-4174-b957-621a0b0b49f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:40.912970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245b409-a1a9-4174-b957-621a0b0b49f6 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:40.913616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245b409-a1a9-4174-b957-621a0b0b49f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:40.913616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245b409-a1a9-4174-b957-621a0b0b49f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:40.953908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245b409-a1a9-4174-b957-621a0b0b49f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:40.955871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245b409-a1a9-4174-b957-621a0b0b49f6 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:40.986625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1053/2101] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:41.975192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f4639-4f2f-4c4b-a24f-d92f2fd63fa9 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:41.976760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-312f4639-4f2f-4c4b-a24f-d92f2fd63fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:42.004202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f4639-4f2f-4c4b-a24f-d92f2fd63fa9 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:42.033553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:42.035344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:42.035832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:42.036411 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] PATH_INFO: `/v3/projects/3cb5eeaf783643c39e3da82beaccc119` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:42.038588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cb5eeaf783643c39e3da82beaccc119)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:42.042745 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:42.292986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cb5eeaf783643c39e3da82beaccc119', 'request_id': 'req-97ee4356-7f4e-4de3-a671-ff92e8060ea2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:42.317304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cb5eeaf783643c39e3da82beaccc119', 'request_id': 'req-97ee4356-7f4e-4de3-a671-ff92e8060ea2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:42.328406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cb5eeaf783643c39e3da82beaccc119', 'request_id': 'req-97ee4356-7f4e-4de3-a671-ff92e8060ea2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:42.434347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] The token cache is being invalidate because project 3cb5eeaf783643c39e3da82beaccc119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:42.435096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:42.436118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:42.439566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1049/2102] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:41 2022] DELETE /identity/v3/projects/3cb5eeaf783643c39e3da82beaccc119 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:42.444297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245b409-a1a9-4174-b957-621a0b0b49f6 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:42.458363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245b409-a1a9-4174-b957-621a0b0b49f6 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:42.560819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:42.561704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:42.561999 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:42.562346 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] PATH_INFO: `/v3/users/fabedb69fe3d428ca16a1d2244ad9481` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:42.563620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fabedb69fe3d428ca16a1d2244ad9481)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:42.569372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:42.570449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:42.825725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fabedb69fe3d428ca16a1d2244ad9481'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:42.851051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fabedb69fe3d428ca16a1d2244ad9481'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:42.858973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fabedb69fe3d428ca16a1d2244ad9481'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:42.953897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fabedb69fe3d428ca16a1d2244ad9481'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:42.955671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4e43a14a-0454-4163-9f1b-76935d08cf27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fabedb69fe3d428ca16a1d2244ad9481'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:42.979104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1054/2103] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:42 2022] DELETE /identity/v3/users/fabedb69fe3d428ca16a1d2244ad9481 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:43.990967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:43.994492 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:44.029670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4356-7f4e-4de3-a671-ff92e8060ea2 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:44.118571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:44.119311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:44.119792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:44.120995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] PATH_INFO: `/v3/projects/b663dad87d35442b86ab77479abcf862` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:44.122892 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b663dad87d35442b86ab77479abcf862)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:44.125776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:44.128883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:44.302513 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b663dad87d35442b86ab77479abcf862', 'request_id': 'req-2ef21474-3d58-4808-878c-a47205862caf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:44.321528 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b663dad87d35442b86ab77479abcf862', 'request_id': 'req-2ef21474-3d58-4808-878c-a47205862caf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:44.328856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b663dad87d35442b86ab77479abcf862', 'request_id': 'req-2ef21474-3d58-4808-878c-a47205862caf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:44.438601 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] The token cache is being invalidate because project b663dad87d35442b86ab77479abcf862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:44.438895 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:44.439521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ef21474-3d58-4808-878c-a47205862caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:44.441131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1050/2104] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:43 2022] DELETE /identity/v3/projects/b663dad87d35442b86ab77479abcf862 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:44.446090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df845df-d300-4cbc-b26f-a38c0c3c7d09 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:44.446657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df845df-d300-4cbc-b26f-a38c0c3c7d09 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:44.446771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df845df-d300-4cbc-b26f-a38c0c3c7d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:44.479722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-1df845df-d300-4cbc-b26f-a38c0c3c7d09 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:44.482230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1df845df-d300-4cbc-b26f-a38c0c3c7d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:44.582858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1055/2105] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:44.583654 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff863fa-1b2b-40b2-8334-c14123c201d2 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:44.584501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff863fa-1b2b-40b2-8334-c14123c201d2 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:44.584783 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff863fa-1b2b-40b2-8334-c14123c201d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:44.617729 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1ff863fa-1b2b-40b2-8334-c14123c201d2 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:44.622535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1ff863fa-1b2b-40b2-8334-c14123c201d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:44.645174 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df845df-d300-4cbc-b26f-a38c0c3c7d09 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:44.654204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1051/2106] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:44.665637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df845df-d300-4cbc-b26f-a38c0c3c7d09 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:44.688652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87120505-7277-4e09-82e9-4384dcd2350b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:44.689807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120505-7277-4e09-82e9-4384dcd2350b admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:44.690283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120505-7277-4e09-82e9-4384dcd2350b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:44.690599 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120505-7277-4e09-82e9-4384dcd2350b admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:44.691362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87120505-7277-4e09-82e9-4384dcd2350b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:44.691665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87120505-7277-4e09-82e9-4384dcd2350b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:44.699578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87120505-7277-4e09-82e9-4384dcd2350b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:44.738528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1056/2107] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:44.741707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff863fa-1b2b-40b2-8334-c14123c201d2 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:44.752907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff863fa-1b2b-40b2-8334-c14123c201d2 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:44.760604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5bb8b-b2c0-4dd6-8194-3333fee38e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:44.761708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5bb8b-b2c0-4dd6-8194-3333fee38e36 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:44.762179 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5bb8b-b2c0-4dd6-8194-3333fee38e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:44.762705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5bb8b-b2c0-4dd6-8194-3333fee38e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:44.763978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5bb8b-b2c0-4dd6-8194-3333fee38e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:44.766277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5bb8b-b2c0-4dd6-8194-3333fee38e36 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:44.978006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1052/2108] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:44 2022] POST /identity/v3/projects => generated 375 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:44.980079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87120505-7277-4e09-82e9-4384dcd2350b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:44.995045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87120505-7277-4e09-82e9-4384dcd2350b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:45.101587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325a314-ec86-4d8b-8ad8-8a5dc6803557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:45.102685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a314-ec86-4d8b-8ad8-8a5dc6803557 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:45.103015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a314-ec86-4d8b-8ad8-8a5dc6803557 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:45.103294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a314-ec86-4d8b-8ad8-8a5dc6803557 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:45.104011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325a314-ec86-4d8b-8ad8-8a5dc6803557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:45.106829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325a314-ec86-4d8b-8ad8-8a5dc6803557 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:45.247223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1057/2109] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:44 2022] POST /identity/v3/users => generated 288 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:45.251180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5bb8b-b2c0-4dd6-8194-3333fee38e36 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:45.265516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5bb8b-b2c0-4dd6-8194-3333fee38e36 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:45.283188 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6054270-b658-4dec-b66f-b55fa98d18bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:45.284150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6054270-b658-4dec-b66f-b55fa98d18bf admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:45.284644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6054270-b658-4dec-b66f-b55fa98d18bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:45.284990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6054270-b658-4dec-b66f-b55fa98d18bf admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:45.285841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6054270-b658-4dec-b66f-b55fa98d18bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:45.289446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6054270-b658-4dec-b66f-b55fa98d18bf admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:45.305575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1053/2110] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:45.306815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325a314-ec86-4d8b-8ad8-8a5dc6803557 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:45.332908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325a314-ec86-4d8b-8ad8-8a5dc6803557 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:45.339355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f995d-4f28-40ed-9f14-d7364e5fcc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:45.339946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f995d-4f28-40ed-9f14-d7364e5fcc62 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:45.340286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f995d-4f28-40ed-9f14-d7364e5fcc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:45.340501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f995d-4f28-40ed-9f14-d7364e5fcc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:45.342412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f995d-4f28-40ed-9f14-d7364e5fcc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:45.344239 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f995d-4f28-40ed-9f14-d7364e5fcc62 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:45.374080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1058/2111] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:45.374418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6054270-b658-4dec-b66f-b55fa98d18bf admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:45.397696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6054270-b658-4dec-b66f-b55fa98d18bf admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:45.410376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb56744-4602-4fed-9529-17c2dbea0afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:45.411332 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb56744-4602-4fed-9529-17c2dbea0afb admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:45.411614 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb56744-4602-4fed-9529-17c2dbea0afb admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:45.411891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb56744-4602-4fed-9529-17c2dbea0afb admin admin] PATH_INFO: `/v3/projects/4b37f3a79d5a46dfa1e7ae5cb76eeac8/users/05b104f8b64f40a69548f05701c4e39f/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:45.442550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb56744-4602-4fed-9529-17c2dbea0afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b37f3a79d5a46dfa1e7ae5cb76eeac8, user_id=05b104f8b64f40a69548f05701c4e39f, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:45.446247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb56744-4602-4fed-9529-17c2dbea0afb admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:45.558530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1054/2112] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:45 2022] PUT /identity/v3/projects/4b37f3a79d5a46dfa1e7ae5cb76eeac8/users/05b104f8b64f40a69548f05701c4e39f/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:45.566830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee870743-1cdc-4198-bc0a-19ae7aaa87df None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:45.567397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee870743-1cdc-4198-bc0a-19ae7aaa87df None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:45.567773 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee870743-1cdc-4198-bc0a-19ae7aaa87df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:45.602705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ee870743-1cdc-4198-bc0a-19ae7aaa87df None None] MFA Rules not processed for user `05b104f8b64f40a69548f05701c4e39f`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:45.604495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ee870743-1cdc-4198-bc0a-19ae7aaa87df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:45.711532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb56744-4602-4fed-9529-17c2dbea0afb admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:45.714114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1059/2113] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:45.803106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb56744-4602-4fed-9529-17c2dbea0afb admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:45.809659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88498a85-37d4-41ef-a185-c67ec03c521d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:45.810755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88498a85-37d4-41ef-a185-c67ec03c521d admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:45.811378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88498a85-37d4-41ef-a185-c67ec03c521d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:45.811808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88498a85-37d4-41ef-a185-c67ec03c521d admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:45.812731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88498a85-37d4-41ef-a185-c67ec03c521d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:45.815193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88498a85-37d4-41ef-a185-c67ec03c521d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:45.952341 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1055/2114] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:45 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:45.956673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee870743-1cdc-4198-bc0a-19ae7aaa87df None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:45.971500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee870743-1cdc-4198-bc0a-19ae7aaa87df None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.053569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:46.078710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff05d7d-81db-4722-9f44-4bf2f9e7dd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:46.079524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff05d7d-81db-4722-9f44-4bf2f9e7dd00 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.079979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff05d7d-81db-4722-9f44-4bf2f9e7dd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.080426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff05d7d-81db-4722-9f44-4bf2f9e7dd00 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.083015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff05d7d-81db-4722-9f44-4bf2f9e7dd00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:46.095805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff05d7d-81db-4722-9f44-4bf2f9e7dd00 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:46.210665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1060/2115] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:45 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:46.212172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88498a85-37d4-41ef-a185-c67ec03c521d admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:46.243351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234c8b7-6c7c-4a61-a9c7-1a680ede6e54 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.244288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234c8b7-6c7c-4a61-a9c7-1a680ede6e54 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.244653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234c8b7-6c7c-4a61-a9c7-1a680ede6e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.247949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88498a85-37d4-41ef-a185-c67ec03c521d admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.266506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6854bc-85c2-43e1-8ea4-e2167c444cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:46.267313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6854bc-85c2-43e1-8ea4-e2167c444cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.267669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6854bc-85c2-43e1-8ea4-e2167c444cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.268024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6854bc-85c2-43e1-8ea4-e2167c444cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.268844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6854bc-85c2-43e1-8ea4-e2167c444cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:46.271089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6854bc-85c2-43e1-8ea4-e2167c444cb8 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:46.294822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-9234c8b7-6c7c-4a61-a9c7-1a680ede6e54 None None] MFA Rules not processed for user `d03b404df6d54b638804804bbc08e1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:46.296144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9234c8b7-6c7c-4a61-a9c7-1a680ede6e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:46.296913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1056/2116] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:46.300708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6854bc-85c2-43e1-8ea4-e2167c444cb8 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:46.313262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6854bc-85c2-43e1-8ea4-e2167c444cb8 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.318155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226958e8-8ae1-4abf-ba59-43fe90cc0a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:46.319502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226958e8-8ae1-4abf-ba59-43fe90cc0a4b admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.319782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226958e8-8ae1-4abf-ba59-43fe90cc0a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.320047 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226958e8-8ae1-4abf-ba59-43fe90cc0a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.320774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226958e8-8ae1-4abf-ba59-43fe90cc0a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:46.322407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226958e8-8ae1-4abf-ba59-43fe90cc0a4b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:46.338153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1057/2117] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:46.339153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226958e8-8ae1-4abf-ba59-43fe90cc0a4b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:46.361422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226958e8-8ae1-4abf-ba59-43fe90cc0a4b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.369128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f0708-cf7e-4658-95b3-af9c18fd564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:46.370070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f0708-cf7e-4658-95b3-af9c18fd564a admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.370591 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f0708-cf7e-4658-95b3-af9c18fd564a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.371587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f0708-cf7e-4658-95b3-af9c18fd564a admin admin] PATH_INFO: `/v3/projects/fc3a43d045da4c478fe850f1a15b45d9/users/953936c9be7d40e39deaa8a5e14b739e/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.397704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f0708-cf7e-4658-95b3-af9c18fd564a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3a43d045da4c478fe850f1a15b45d9, user_id=953936c9be7d40e39deaa8a5e14b739e, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:46.400888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f0708-cf7e-4658-95b3-af9c18fd564a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:46.408036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1061/2118] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:46.409464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b9dfc-37ed-442c-ae38-b8300a24c0fd None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.409898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b9dfc-37ed-442c-ae38-b8300a24c0fd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.410245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b9dfc-37ed-442c-ae38-b8300a24c0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.432801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e72b9dfc-37ed-442c-ae38-b8300a24c0fd None None] MFA Rules not processed for user `6dc49525e3394668b4f1610c4badc144`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:46.434084 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e72b9dfc-37ed-442c-ae38-b8300a24c0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:46.480295 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1058/2119] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:46 2022] PUT /identity/v3/projects/fc3a43d045da4c478fe850f1a15b45d9/users/953936c9be7d40e39deaa8a5e14b739e/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:46.484658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f0708-cf7e-4658-95b3-af9c18fd564a admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:46.484658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a47f0708-cf7e-4658-95b3-af9c18fd564a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:46.528082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1062/2120] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:46.538358 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb397efc-ff10-48fb-ae78-47bf7b9aca37 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.538904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb397efc-ff10-48fb-ae78-47bf7b9aca37 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.539374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb397efc-ff10-48fb-ae78-47bf7b9aca37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.576349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-cb397efc-ff10-48fb-ae78-47bf7b9aca37 None None] MFA Rules not processed for user `953936c9be7d40e39deaa8a5e14b739e`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:46.577580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-cb397efc-ff10-48fb-ae78-47bf7b9aca37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:46.625168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f0708-cf7e-4658-95b3-af9c18fd564a admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.638598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983ca40-178c-4c96-97d4-2deff137542e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:46.639331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983ca40-178c-4c96-97d4-2deff137542e admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.639577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983ca40-178c-4c96-97d4-2deff137542e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.640254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983ca40-178c-4c96-97d4-2deff137542e admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.640571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983ca40-178c-4c96-97d4-2deff137542e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:46.642890 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983ca40-178c-4c96-97d4-2deff137542e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:46.665706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1063/2121] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:46.666864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb397efc-ff10-48fb-ae78-47bf7b9aca37 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:46.680274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb397efc-ff10-48fb-ae78-47bf7b9aca37 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.686951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2f59c-8013-40f7-b795-025dedb9447c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:46.701658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2f59c-8013-40f7-b795-025dedb9447c admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.701658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2f59c-8013-40f7-b795-025dedb9447c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.701658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2f59c-8013-40f7-b795-025dedb9447c admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.701658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2f59c-8013-40f7-b795-025dedb9447c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:46.701658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2f59c-8013-40f7-b795-025dedb9447c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:46.793980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1059/2122] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:46 2022] POST /identity/v3/projects => generated 373 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:46.797918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983ca40-178c-4c96-97d4-2deff137542e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:46.825036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1064/2123] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:46 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:46.826818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2f59c-8013-40f7-b795-025dedb9447c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:46.925307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2f59c-8013-40f7-b795-025dedb9447c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.930641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fb163-87b3-4023-a67b-99af5179c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:46.931616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fb163-87b3-4023-a67b-99af5179c9ee admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.931886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fb163-87b3-4023-a67b-99af5179c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.932262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fb163-87b3-4023-a67b-99af5179c9ee admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.933082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59fb163-87b3-4023-a67b-99af5179c9ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:46.933984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983ca40-178c-4c96-97d4-2deff137542e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.935347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59fb163-87b3-4023-a67b-99af5179c9ee admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:46.940217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b4fd6-eaef-41a5-a1c7-8c856e906c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:46.940912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b4fd6-eaef-41a5-a1c7-8c856e906c93 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.941966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b4fd6-eaef-41a5-a1c7-8c856e906c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.942301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b4fd6-eaef-41a5-a1c7-8c856e906c93 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.943407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b4fd6-eaef-41a5-a1c7-8c856e906c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:46.946726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b4fd6-eaef-41a5-a1c7-8c856e906c93 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.035716 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1065/2124] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:46 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:47.036988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1060/2125] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:46 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:47.042168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fb163-87b3-4023-a67b-99af5179c9ee admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.043588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b4fd6-eaef-41a5-a1c7-8c856e906c93 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.053404 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fb163-87b3-4023-a67b-99af5179c9ee admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.056444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b4fd6-eaef-41a5-a1c7-8c856e906c93 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.059032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee589ce-6a3e-4a31-aa43-a5b08740c241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.060629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee589ce-6a3e-4a31-aa43-a5b08740c241 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.060629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee589ce-6a3e-4a31-aa43-a5b08740c241 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.060924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee589ce-6a3e-4a31-aa43-a5b08740c241 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.061634 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee589ce-6a3e-4a31-aa43-a5b08740c241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.063741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee589ce-6a3e-4a31-aa43-a5b08740c241 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.064220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2875a-7914-40fa-9a36-5bcfb32d825d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.065142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2875a-7914-40fa-9a36-5bcfb32d825d admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.065582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2875a-7914-40fa-9a36-5bcfb32d825d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.066016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2875a-7914-40fa-9a36-5bcfb32d825d admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.067253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2875a-7914-40fa-9a36-5bcfb32d825d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.069218 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2875a-7914-40fa-9a36-5bcfb32d825d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.079950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1066/2126] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:47.080881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee589ce-6a3e-4a31-aa43-a5b08740c241 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.090938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1061/2127] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:47.096107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee589ce-6a3e-4a31-aa43-a5b08740c241 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.101017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2875a-7914-40fa-9a36-5bcfb32d825d admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.103427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fe786-bb96-4ada-9c56-db603f60207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.104447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fe786-bb96-4ada-9c56-db603f60207f admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.104867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fe786-bb96-4ada-9c56-db603f60207f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.105249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fe786-bb96-4ada-9c56-db603f60207f admin admin] PATH_INFO: `/v3/projects/f339b1033b2948bea1d9fe6eb1f387b9/users/3b52c97f202c4bff83ec4875658b6e5b/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.119862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2875a-7914-40fa-9a36-5bcfb32d825d admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.126812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c043340-f778-4b70-9a77-6c21bb46830b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.127680 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c043340-f778-4b70-9a77-6c21bb46830b admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.128032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c043340-f778-4b70-9a77-6c21bb46830b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.128313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c043340-f778-4b70-9a77-6c21bb46830b admin admin] PATH_INFO: `/v3/projects/e083c62211f5411e84a7e154e1e9a4d4/users/752b3fee1f08439f9c7dc2d418b04e7d/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.128859 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fe786-bb96-4ada-9c56-db603f60207f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f339b1033b2948bea1d9fe6eb1f387b9, user_id=3b52c97f202c4bff83ec4875658b6e5b, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.132827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fe786-bb96-4ada-9c56-db603f60207f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.157635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c043340-f778-4b70-9a77-6c21bb46830b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e083c62211f5411e84a7e154e1e9a4d4, user_id=752b3fee1f08439f9c7dc2d418b04e7d, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.161454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c043340-f778-4b70-9a77-6c21bb46830b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.190220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1067/2128] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:47 2022] PUT /identity/v3/projects/f339b1033b2948bea1d9fe6eb1f387b9/users/3b52c97f202c4bff83ec4875658b6e5b/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:47.194860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fe786-bb96-4ada-9c56-db603f60207f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.246593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1062/2129] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:47 2022] PUT /identity/v3/projects/e083c62211f5411e84a7e154e1e9a4d4/users/752b3fee1f08439f9c7dc2d418b04e7d/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:47.250380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c043340-f778-4b70-9a77-6c21bb46830b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.330552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c043340-f778-4b70-9a77-6c21bb46830b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.354272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fe786-bb96-4ada-9c56-db603f60207f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.355801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454c532-c9a5-4d1e-aef4-7d5783f8d8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.358694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454c532-c9a5-4d1e-aef4-7d5783f8d8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.359080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454c532-c9a5-4d1e-aef4-7d5783f8d8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.359461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454c532-c9a5-4d1e-aef4-7d5783f8d8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.360699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368909ae-0150-4448-b467-a547873cc8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.361197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454c532-c9a5-4d1e-aef4-7d5783f8d8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.362019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368909ae-0150-4448-b467-a547873cc8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.362318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368909ae-0150-4448-b467-a547873cc8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.362674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368909ae-0150-4448-b467-a547873cc8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.363507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368909ae-0150-4448-b467-a547873cc8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.365133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368909ae-0150-4448-b467-a547873cc8e0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.366082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454c532-c9a5-4d1e-aef4-7d5783f8d8c4 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.401579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1063/2130] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:47.401943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1068/2131] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:47.402839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368909ae-0150-4448-b467-a547873cc8e0 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.408343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454c532-c9a5-4d1e-aef4-7d5783f8d8c4 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.417093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368909ae-0150-4448-b467-a547873cc8e0 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.445024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454c532-c9a5-4d1e-aef4-7d5783f8d8c4 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.447479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304174e-7253-406e-b120-d8143267d005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.448400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304174e-7253-406e-b120-d8143267d005 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.448684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304174e-7253-406e-b120-d8143267d005 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.449027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304174e-7253-406e-b120-d8143267d005 admin admin] PATH_INFO: `/v3/projects/f339b1033b2948bea1d9fe6eb1f387b9/users/3b52c97f202c4bff83ec4875658b6e5b/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.452004 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:47.452971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ddce6-7e3c-45ff-9d27-6f1e08665d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.454001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304174e-7253-406e-b120-d8143267d005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f339b1033b2948bea1d9fe6eb1f387b9, user_id=3b52c97f202c4bff83ec4875658b6e5b, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.454126 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ddce6-7e3c-45ff-9d27-6f1e08665d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.454384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ddce6-7e3c-45ff-9d27-6f1e08665d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.454720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ddce6-7e3c-45ff-9d27-6f1e08665d57 admin admin] PATH_INFO: `/v3/projects/e083c62211f5411e84a7e154e1e9a4d4/users/752b3fee1f08439f9c7dc2d418b04e7d/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.456963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ddce6-7e3c-45ff-9d27-6f1e08665d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e083c62211f5411e84a7e154e1e9a4d4, user_id=752b3fee1f08439f9c7dc2d418b04e7d, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.462050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304174e-7253-406e-b120-d8143267d005 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.462151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ddce6-7e3c-45ff-9d27-6f1e08665d57 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.482948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1064/2132] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:47 2022] PUT /identity/v3/projects/e083c62211f5411e84a7e154e1e9a4d4/users/752b3fee1f08439f9c7dc2d418b04e7d/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:47.488478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5579ab-8171-4958-aa5c-27fc3ef8f993 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.488763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5579ab-8171-4958-aa5c-27fc3ef8f993 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.489039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5579ab-8171-4958-aa5c-27fc3ef8f993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.493654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1069/2133] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:47 2022] PUT /identity/v3/projects/f339b1033b2948bea1d9fe6eb1f387b9/users/3b52c97f202c4bff83ec4875658b6e5b/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:47.528636 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4e5579ab-8171-4958-aa5c-27fc3ef8f993 None None] MFA Rules not processed for user `752b3fee1f08439f9c7dc2d418b04e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:47.530857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4e5579ab-8171-4958-aa5c-27fc3ef8f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:47.628283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1065/2134] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:47.642583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304174e-7253-406e-b120-d8143267d005 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.645171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0304174e-7253-406e-b120-d8143267d005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:47.795644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304174e-7253-406e-b120-d8143267d005 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.807723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b6b23-9a95-440b-8cd0-79cc095fa5b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.808836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b6b23-9a95-440b-8cd0-79cc095fa5b8 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.809373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b6b23-9a95-440b-8cd0-79cc095fa5b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.810342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b6b23-9a95-440b-8cd0-79cc095fa5b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.823610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b6b23-9a95-440b-8cd0-79cc095fa5b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.827669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b6b23-9a95-440b-8cd0-79cc095fa5b8 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.839875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1070/2135] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:48.041251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5579ab-8171-4958-aa5c-27fc3ef8f993 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:48.042881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4e5579ab-8171-4958-aa5c-27fc3ef8f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:48.058217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5579ab-8171-4958-aa5c-27fc3ef8f993 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:48.061621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acd889-2de3-4ed6-a818-852dfd581417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:48.062307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acd889-2de3-4ed6-a818-852dfd581417 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:48.062626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acd889-2de3-4ed6-a818-852dfd581417 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:48.062903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acd889-2de3-4ed6-a818-852dfd581417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:48.127602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36acd889-2de3-4ed6-a818-852dfd581417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:48.129210 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36acd889-2de3-4ed6-a818-852dfd581417 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:48.139582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1066/2136] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:48.365144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:50.492988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:50.679766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:53.940869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b6b23-9a95-440b-8cd0-79cc095fa5b8 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:54.026252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:54.044107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b6b23-9a95-440b-8cd0-79cc095fa5b8 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:54.049781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:54.054566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:54.054566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:54.054566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] PATH_INFO: `/v3/users/05b104f8b64f40a69548f05701c4e39f` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:54.056750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b104f8b64f40a69548f05701c4e39f)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:54.058624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:54.340319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b104f8b64f40a69548f05701c4e39f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.353883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b104f8b64f40a69548f05701c4e39f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.356842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b104f8b64f40a69548f05701c4e39f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.473588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b104f8b64f40a69548f05701c4e39f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.474915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b104f8b64f40a69548f05701c4e39f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.511892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1071/2137] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:53 2022] DELETE /identity/v3/users/05b104f8b64f40a69548f05701c4e39f => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:54.516402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575311a9-5de4-45b2-bcd0-bda1d35a6266 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:54.516879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575311a9-5de4-45b2-bcd0-bda1d35a6266 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:54.516975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575311a9-5de4-45b2-bcd0-bda1d35a6266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:54.546567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-575311a9-5de4-45b2-bcd0-bda1d35a6266 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:54.546567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-575311a9-5de4-45b2-bcd0-bda1d35a6266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:54.559875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:54.630351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1067/2138] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:54.640330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:54.641881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:54.722909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe343c-0f74-452a-a790-850dcfd31f38 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:54.727319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519e1fd-d2ff-4783-b92b-9304fb5c6e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:54.727925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519e1fd-d2ff-4783-b92b-9304fb5c6e84 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:54.728212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519e1fd-d2ff-4783-b92b-9304fb5c6e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:54.728537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519e1fd-d2ff-4783-b92b-9304fb5c6e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:54.738662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519e1fd-d2ff-4783-b92b-9304fb5c6e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:54.740847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519e1fd-d2ff-4783-b92b-9304fb5c6e84 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:54.757330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1072/2139] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:54.861026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde9307-b763-4bf4-8f5e-e676f4e7286f None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:54.864072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde9307-b763-4bf4-8f5e-e676f4e7286f None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:54.864072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde9307-b763-4bf4-8f5e-e676f4e7286f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:54.881614 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-ffde9307-b763-4bf4-8f5e-e676f4e7286f None None] MFA Rules not processed for user `a01061da6fc0404facce47e669ec80af`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:54.882885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ffde9307-b763-4bf4-8f5e-e676f4e7286f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:54.990495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1068/2140] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:56.292525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519e1fd-d2ff-4783-b92b-9304fb5c6e84 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:56.293478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8519e1fd-d2ff-4783-b92b-9304fb5c6e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:56.313398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519e1fd-d2ff-4783-b92b-9304fb5c6e84 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:56.318543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:56.319575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:56.319758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:56.320041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] PATH_INFO: `/v3/projects/4b37f3a79d5a46dfa1e7ae5cb76eeac8` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:56.321509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b37f3a79d5a46dfa1e7ae5cb76eeac8)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:56.323727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:56.451475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b37f3a79d5a46dfa1e7ae5cb76eeac8', 'request_id': 'req-da93ca7f-4c9f-4439-9397-2133c8317543'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:56.466927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b37f3a79d5a46dfa1e7ae5cb76eeac8', 'request_id': 'req-da93ca7f-4c9f-4439-9397-2133c8317543'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:56.488155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b37f3a79d5a46dfa1e7ae5cb76eeac8', 'request_id': 'req-da93ca7f-4c9f-4439-9397-2133c8317543'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:56.497064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:56.565529 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] The token cache is being invalidate because project 4b37f3a79d5a46dfa1e7ae5cb76eeac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:56.573282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:56.573282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-da93ca7f-4c9f-4439-9397-2133c8317543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:56.573282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1073/2141] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:56 2022] DELETE /identity/v3/projects/4b37f3a79d5a46dfa1e7ae5cb76eeac8 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:56.573935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde9307-b763-4bf4-8f5e-e676f4e7286f None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:56.591316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde9307-b763-4bf4-8f5e-e676f4e7286f None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:56.667949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:56.669133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:56.669517 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:56.669872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] PATH_INFO: `/v3/users/953936c9be7d40e39deaa8a5e14b739e` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:56.687050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] RBAC: Authorizing `identity:delete_user(user_id=953936c9be7d40e39deaa8a5e14b739e)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:56.687050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:56.889211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953936c9be7d40e39deaa8a5e14b739e'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:56.899543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953936c9be7d40e39deaa8a5e14b739e'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:56.918731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:57.021913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953936c9be7d40e39deaa8a5e14b739e'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:57.022902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953936c9be7d40e39deaa8a5e14b739e'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:57.023671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953936c9be7d40e39deaa8a5e14b739e'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:57.046385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1069/2142] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:56 2022] DELETE /identity/v3/users/953936c9be7d40e39deaa8a5e14b739e => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:57.183154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14758ac4-0e00-4246-830b-d313b04a2ab7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:57.183741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14758ac4-0e00-4246-830b-d313b04a2ab7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:57.184118 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14758ac4-0e00-4246-830b-d313b04a2ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:57.217765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-14758ac4-0e00-4246-830b-d313b04a2ab7 None None] MFA Rules not processed for user `3b52c97f202c4bff83ec4875658b6e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:57.218607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-14758ac4-0e00-4246-830b-d313b04a2ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:57.318724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1074/2143] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:00:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:57.430662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:57.432311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:57.561829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083134e4-d191-4924-849c-97dfe1d9459f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:57.569433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89902799-4405-4e18-b712-be72302bff98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:57.570240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89902799-4405-4e18-b712-be72302bff98 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:57.570761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89902799-4405-4e18-b712-be72302bff98 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:57.571138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89902799-4405-4e18-b712-be72302bff98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:57.573184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-89902799-4405-4e18-b712-be72302bff98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:57.656368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89902799-4405-4e18-b712-be72302bff98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:57.658302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89902799-4405-4e18-b712-be72302bff98 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:57.671426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1070/2144] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:57.687596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14758ac4-0e00-4246-830b-d313b04a2ab7 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:57.689512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-14758ac4-0e00-4246-830b-d313b04a2ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:57.805271 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14758ac4-0e00-4246-830b-d313b04a2ab7 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:57.810959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:57.811879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:57.812261 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:57.812638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] PATH_INFO: `/v3/projects/fc3a43d045da4c478fe850f1a15b45d9` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:57.813972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3a43d045da4c478fe850f1a15b45d9)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:57.816023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:57.937642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3a43d045da4c478fe850f1a15b45d9', 'request_id': 'req-b158889c-9f3f-4a58-acbc-3b89e76ecaca'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:57.944721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3a43d045da4c478fe850f1a15b45d9', 'request_id': 'req-b158889c-9f3f-4a58-acbc-3b89e76ecaca'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:57.950932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3a43d045da4c478fe850f1a15b45d9', 'request_id': 'req-b158889c-9f3f-4a58-acbc-3b89e76ecaca'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:58.101155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] The token cache is being invalidate because project fc3a43d045da4c478fe850f1a15b45d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:58.101709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:58.102924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:58.105123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1075/2145] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:57 2022] DELETE /identity/v3/projects/fc3a43d045da4c478fe850f1a15b45d9 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:58.109345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89902799-4405-4e18-b712-be72302bff98 service nova] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:58.126608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89902799-4405-4e18-b712-be72302bff98 service nova] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:58.226576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:58.227566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:58.228084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:58.228981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] PATH_INFO: `/v3/users/752b3fee1f08439f9c7dc2d418b04e7d` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:58.230567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=752b3fee1f08439f9c7dc2d418b04e7d)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:58.233945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:58.262909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:58.438531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752b3fee1f08439f9c7dc2d418b04e7d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:58.456556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752b3fee1f08439f9c7dc2d418b04e7d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:58.560045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752b3fee1f08439f9c7dc2d418b04e7d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:58.560677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752b3fee1f08439f9c7dc2d418b04e7d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:58.561387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-bbf05d08-1a6f-4f56-b6a9-f9c915a3b7b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752b3fee1f08439f9c7dc2d418b04e7d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:58.594974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1071/2146] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:58 2022] DELETE /identity/v3/users/752b3fee1f08439f9c7dc2d418b04e7d => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:59.555543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:59.557275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:59.594854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158889c-9f3f-4a58-acbc-3b89e76ecaca admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:59.615897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:59.793323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:59.797269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:59.797604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:59.797892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] PATH_INFO: `/v3/projects/e083c62211f5411e84a7e154e1e9a4d4` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:59.800234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e083c62211f5411e84a7e154e1e9a4d4)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:59.802732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:00.071842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e083c62211f5411e84a7e154e1e9a4d4', 'request_id': 'req-eb9c1513-fada-40de-8a7d-6aeee3de075a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.088348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e083c62211f5411e84a7e154e1e9a4d4', 'request_id': 'req-eb9c1513-fada-40de-8a7d-6aeee3de075a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.096941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e083c62211f5411e84a7e154e1e9a4d4', 'request_id': 'req-eb9c1513-fada-40de-8a7d-6aeee3de075a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.249089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] The token cache is being invalidate because project e083c62211f5411e84a7e154e1e9a4d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:00.249479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.250246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-eb9c1513-fada-40de-8a7d-6aeee3de075a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.252546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1076/2147] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:59 2022] DELETE /identity/v3/projects/e083c62211f5411e84a7e154e1e9a4d4 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:00.259550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4651-489a-462a-871a-a4a83e319145 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.260104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4651-489a-462a-871a-a4a83e319145 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.260297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4651-489a-462a-871a-a4a83e319145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.285359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-735c4651-489a-462a-871a-a4a83e319145 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:00.292632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-735c4651-489a-462a-871a-a4a83e319145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:00.382187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1072/2148] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:00.391534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40508fc5-b7ad-4338-a322-5adebfadc2ff None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.391820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40508fc5-b7ad-4338-a322-5adebfadc2ff None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.392147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40508fc5-b7ad-4338-a322-5adebfadc2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.437428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-40508fc5-b7ad-4338-a322-5adebfadc2ff None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:00.438708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-40508fc5-b7ad-4338-a322-5adebfadc2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:00.460335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1077/2149] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:00.464655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c4651-489a-462a-871a-a4a83e319145 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:00.478095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c4651-489a-462a-871a-a4a83e319145 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:00.483036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c7f8b-ad17-4594-be1c-f9489a0050c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:00.483726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c7f8b-ad17-4594-be1c-f9489a0050c8 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.484012 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c7f8b-ad17-4594-be1c-f9489a0050c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.484296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c7f8b-ad17-4594-be1c-f9489a0050c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.485067 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c7f8b-ad17-4594-be1c-f9489a0050c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:00.485402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c7f8b-ad17-4594-be1c-f9489a0050c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:00.487044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c7f8b-ad17-4594-be1c-f9489a0050c8 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:00.502421 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1073/2150] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:01:00.507154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40508fc5-b7ad-4338-a322-5adebfadc2ff None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:00.523516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40508fc5-b7ad-4338-a322-5adebfadc2ff None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:00.530321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df2782-b13d-4ee6-9da4-8dae954b03db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:00.531127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df2782-b13d-4ee6-9da4-8dae954b03db admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.531778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df2782-b13d-4ee6-9da4-8dae954b03db admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.531907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df2782-b13d-4ee6-9da4-8dae954b03db admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.532805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df2782-b13d-4ee6-9da4-8dae954b03db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:00.539526 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df2782-b13d-4ee6-9da4-8dae954b03db admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:00.578321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4ba45-0090-4259-a7d3-ad646ea286a8 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.578684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4ba45-0090-4259-a7d3-ad646ea286a8 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.578977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4ba45-0090-4259-a7d3-ad646ea286a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.599591 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-eff4ba45-0090-4259-a7d3-ad646ea286a8 None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:00.600854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-eff4ba45-0090-4259-a7d3-ad646ea286a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:00.669002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1078/2151] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:00 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:00.670882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df2782-b13d-4ee6-9da4-8dae954b03db admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:00.714894 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1074/2152] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 06:01:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:00.781161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4ba45-0090-4259-a7d3-ad646ea286a8 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:00.782666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-eff4ba45-0090-4259-a7d3-ad646ea286a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:00.797735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df2782-b13d-4ee6-9da4-8dae954b03db admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:00.811216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0d98b-f133-4922-b59e-1e0e12533e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:00.811920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0d98b-f133-4922-b59e-1e0e12533e42 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.812202 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0d98b-f133-4922-b59e-1e0e12533e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.812481 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0d98b-f133-4922-b59e-1e0e12533e42 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.813183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0d98b-f133-4922-b59e-1e0e12533e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:00.820146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0d98b-f133-4922-b59e-1e0e12533e42 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:00.850275 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:00.936518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4ba45-0090-4259-a7d3-ad646ea286a8 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:00.970739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1079/2153] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:00 2022] POST /identity/v3/users => generated 287 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:00.971815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0d98b-f133-4922-b59e-1e0e12533e42 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.003987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0d98b-f133-4922-b59e-1e0e12533e42 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.026917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93169b42-8661-4946-bc30-5ed004351766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.027621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93169b42-8661-4946-bc30-5ed004351766 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.027907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93169b42-8661-4946-bc30-5ed004351766 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.028387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93169b42-8661-4946-bc30-5ed004351766 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.029137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93169b42-8661-4946-bc30-5ed004351766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.030788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93169b42-8661-4946-bc30-5ed004351766 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.054368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1080/2154] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:01.056692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93169b42-8661-4946-bc30-5ed004351766 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.074625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93169b42-8661-4946-bc30-5ed004351766 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.074725 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0e94b-8fcb-47f4-883e-c0bf2ad50e2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.075566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0e94b-8fcb-47f4-883e-c0bf2ad50e2a service swift] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.075854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0e94b-8fcb-47f4-883e-c0bf2ad50e2a service swift] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.076129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0e94b-8fcb-47f4-883e-c0bf2ad50e2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.082613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ddd9d-ed32-4b9a-a170-ec38e40a86df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.083622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ddd9d-ed32-4b9a-a170-ec38e40a86df admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.086768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ddd9d-ed32-4b9a-a170-ec38e40a86df admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.086768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ddd9d-ed32-4b9a-a170-ec38e40a86df admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.086768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ddd9d-ed32-4b9a-a170-ec38e40a86df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.089245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ddd9d-ed32-4b9a-a170-ec38e40a86df admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.122369 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1081/2155] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:01.125545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ddd9d-ed32-4b9a-a170-ec38e40a86df admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.149931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ddd9d-ed32-4b9a-a170-ec38e40a86df admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.156587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6b53a-4a7b-49f2-8d53-b0a8ec89dce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.157712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6b53a-4a7b-49f2-8d53-b0a8ec89dce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.158305 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6b53a-4a7b-49f2-8d53-b0a8ec89dce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.158838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6b53a-4a7b-49f2-8d53-b0a8ec89dce0 admin admin] PATH_INFO: `/v3/projects/2c8065704a2d4030bbce3a2bf4d6caae/users/900bd6e65c9146c5b9b9621d11751753/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.196446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6b53a-4a7b-49f2-8d53-b0a8ec89dce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8065704a2d4030bbce3a2bf4d6caae, user_id=900bd6e65c9146c5b9b9621d11751753, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.199711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6b53a-4a7b-49f2-8d53-b0a8ec89dce0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.210529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0e94b-8fcb-47f4-883e-c0bf2ad50e2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.212507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0e94b-8fcb-47f4-883e-c0bf2ad50e2a service swift] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.223382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1075/2156] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:01:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:01:01.321051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1082/2157] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:01 2022] PUT /identity/v3/projects/2c8065704a2d4030bbce3a2bf4d6caae/users/900bd6e65c9146c5b9b9621d11751753/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:01.332931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badb673-dfec-46cb-85ed-67f8b82a9559 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.333930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badb673-dfec-46cb-85ed-67f8b82a9559 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.334359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badb673-dfec-46cb-85ed-67f8b82a9559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.399821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1badb673-dfec-46cb-85ed-67f8b82a9559 None None] MFA Rules not processed for user `900bd6e65c9146c5b9b9621d11751753`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:01.401507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1badb673-dfec-46cb-85ed-67f8b82a9559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:01.570368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6b53a-4a7b-49f2-8d53-b0a8ec89dce0 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.570813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1076/2158] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:01.667865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6b53a-4a7b-49f2-8d53-b0a8ec89dce0 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.675539 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a22657-8f84-43c1-94c4-3d9fc6981c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.676369 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a22657-8f84-43c1-94c4-3d9fc6981c96 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.676658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a22657-8f84-43c1-94c4-3d9fc6981c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.676931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a22657-8f84-43c1-94c4-3d9fc6981c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.678085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a22657-8f84-43c1-94c4-3d9fc6981c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.680144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a22657-8f84-43c1-94c4-3d9fc6981c96 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.961315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1083/2159] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:01 2022] POST /identity/v3/projects => generated 373 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:01.965169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badb673-dfec-46cb-85ed-67f8b82a9559 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.981684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badb673-dfec-46cb-85ed-67f8b82a9559 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:02.098016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4df8a9-796b-4620-84af-a9a1c040b570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:02.098844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4df8a9-796b-4620-84af-a9a1c040b570 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:02.099240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4df8a9-796b-4620-84af-a9a1c040b570 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:02.099599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4df8a9-796b-4620-84af-a9a1c040b570 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:02.100412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4df8a9-796b-4620-84af-a9a1c040b570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:02.106705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4df8a9-796b-4620-84af-a9a1c040b570 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:02.245962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a22657-8f84-43c1-94c4-3d9fc6981c96 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:02.253145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1077/2160] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:01 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:02.259927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a22657-8f84-43c1-94c4-3d9fc6981c96 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:02.300920 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9a49-9616-47af-8eb0-9853934d9453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:02.302012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9a49-9616-47af-8eb0-9853934d9453 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:02.303070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9a49-9616-47af-8eb0-9853934d9453 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:02.303462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9a49-9616-47af-8eb0-9853934d9453 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:02.304331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b9a49-9616-47af-8eb0-9853934d9453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:02.306037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b9a49-9616-47af-8eb0-9853934d9453 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:02.357434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1084/2161] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:02.361873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4df8a9-796b-4620-84af-a9a1c040b570 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:02.374923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4df8a9-796b-4620-84af-a9a1c040b570 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:02.388002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f1e83-a7e3-4ffa-988d-685861840cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:02.389451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f1e83-a7e3-4ffa-988d-685861840cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:02.389451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f1e83-a7e3-4ffa-988d-685861840cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:02.389451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f1e83-a7e3-4ffa-988d-685861840cd5 admin admin] PATH_INFO: `/v3/projects/322da25f773f4d4eaf73981a98ff6a77/users/7ae95f2764c64f928e71adfa196cfb62/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:02.417717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f1e83-a7e3-4ffa-988d-685861840cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=322da25f773f4d4eaf73981a98ff6a77, user_id=7ae95f2764c64f928e71adfa196cfb62, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:02.420590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f1e83-a7e3-4ffa-988d-685861840cd5 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:02.984382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9a49-9616-47af-8eb0-9853934d9453 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:02.986364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1078/2162] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:02 2022] PUT /identity/v3/projects/322da25f773f4d4eaf73981a98ff6a77/users/7ae95f2764c64f928e71adfa196cfb62/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:03.004772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9a49-9616-47af-8eb0-9853934d9453 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:03.086060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:03.090307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399583a1-73bb-43bc-85c3-3fb26413a917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:03.090307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399583a1-73bb-43bc-85c3-3fb26413a917 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:03.090307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399583a1-73bb-43bc-85c3-3fb26413a917 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:03.090307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399583a1-73bb-43bc-85c3-3fb26413a917 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:03.090737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399583a1-73bb-43bc-85c3-3fb26413a917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:03.092908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399583a1-73bb-43bc-85c3-3fb26413a917 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:03.111961 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f1e83-a7e3-4ffa-988d-685861840cd5 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:03.112683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1085/2163] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:03.124128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f1e83-a7e3-4ffa-988d-685861840cd5 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:03.131759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7115e-0518-4566-b1a2-a58d6e3ec525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:03.133286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7115e-0518-4566-b1a2-a58d6e3ec525 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:03.133286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7115e-0518-4566-b1a2-a58d6e3ec525 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:03.134540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7115e-0518-4566-b1a2-a58d6e3ec525 admin admin] PATH_INFO: `/v3/projects/322da25f773f4d4eaf73981a98ff6a77/users/7ae95f2764c64f928e71adfa196cfb62/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:03.137059 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7115e-0518-4566-b1a2-a58d6e3ec525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=322da25f773f4d4eaf73981a98ff6a77, user_id=7ae95f2764c64f928e71adfa196cfb62, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:03.139580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7115e-0518-4566-b1a2-a58d6e3ec525 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:03.176946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b926f3-e746-429f-a914-e49d076c83df None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:03.177351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b926f3-e746-429f-a914-e49d076c83df None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:03.177597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b926f3-e746-429f-a914-e49d076c83df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:03.179070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1079/2164] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:03 2022] PUT /identity/v3/projects/322da25f773f4d4eaf73981a98ff6a77/users/7ae95f2764c64f928e71adfa196cfb62/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:03.209563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-48b926f3-e746-429f-a914-e49d076c83df None None] MFA Rules not processed for user `7ae95f2764c64f928e71adfa196cfb62`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:03.210845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-48b926f3-e746-429f-a914-e49d076c83df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:03.298290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1086/2165] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:03.306612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7115e-0518-4566-b1a2-a58d6e3ec525 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:03.309317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d2c7115e-0518-4566-b1a2-a58d6e3ec525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:03.387911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7115e-0518-4566-b1a2-a58d6e3ec525 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:03.391824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb080ab-8ea3-4089-b7bb-437a315834e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:03.392617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb080ab-8ea3-4089-b7bb-437a315834e7 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:03.392999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb080ab-8ea3-4089-b7bb-437a315834e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:03.393344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb080ab-8ea3-4089-b7bb-437a315834e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:03.469150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb080ab-8ea3-4089-b7bb-437a315834e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:03.470770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb080ab-8ea3-4089-b7bb-437a315834e7 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:03.481455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1080/2166] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:03.651438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e121af-e2cb-44ac-bdd8-7a15547fd8f6 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:03.651728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e121af-e2cb-44ac-bdd8-7a15547fd8f6 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:03.652182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e121af-e2cb-44ac-bdd8-7a15547fd8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:03.695644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-39e121af-e2cb-44ac-bdd8-7a15547fd8f6 None None] MFA Rules not processed for user `d03b404df6d54b638804804bbc08e1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:03.701435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-39e121af-e2cb-44ac-bdd8-7a15547fd8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:03.803350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1087/2167] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:03.856415 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:05.034269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:08.209081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb080ab-8ea3-4089-b7bb-437a315834e7 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:08.219771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:08.340281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb080ab-8ea3-4089-b7bb-437a315834e7 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:08.349532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:08.350438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.350760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.353417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] PATH_INFO: `/v3/users/900bd6e65c9146c5b9b9621d11751753` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.353417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=900bd6e65c9146c5b9b9621d11751753)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:08.354958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:08.546399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '900bd6e65c9146c5b9b9621d11751753'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:08.557125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '900bd6e65c9146c5b9b9621d11751753'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:08.600653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d270cb-c1a8-427a-a5b3-7fb5e90c88dc None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.600653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d270cb-c1a8-427a-a5b3-7fb5e90c88dc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.600653 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d270cb-c1a8-427a-a5b3-7fb5e90c88dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.624587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-68d270cb-c1a8-427a-a5b3-7fb5e90c88dc None None] MFA Rules not processed for user `13e5d648af294c7cbc86ccd362914728`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:08.626248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-68d270cb-c1a8-427a-a5b3-7fb5e90c88dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:08.663850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '900bd6e65c9146c5b9b9621d11751753'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:08.664782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '900bd6e65c9146c5b9b9621d11751753'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:08.665877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '900bd6e65c9146c5b9b9621d11751753'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:08.709542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1088/2168] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:08.709696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1081/2169] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:08 2022] DELETE /identity/v3/users/900bd6e65c9146c5b9b9621d11751753 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:08.713144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527fb01-4d8e-4906-bd7c-95bc185b8202 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.713591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527fb01-4d8e-4906-bd7c-95bc185b8202 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.713884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527fb01-4d8e-4906-bd7c-95bc185b8202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.739078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-1527fb01-4d8e-4906-bd7c-95bc185b8202 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:08.741820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1527fb01-4d8e-4906-bd7c-95bc185b8202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:08.827885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1089/2170] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:08.841796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:08.843362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:08.978706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049904e-0d0e-493a-979c-fde601b91c33 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:08.986839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50532e-adb2-4978-9c38-f85d6b5a6e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:08.987762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50532e-adb2-4978-9c38-f85d6b5a6e55 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.988278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50532e-adb2-4978-9c38-f85d6b5a6e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.988803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50532e-adb2-4978-9c38-f85d6b5a6e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:09.008398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50532e-adb2-4978-9c38-f85d6b5a6e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:09.010558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50532e-adb2-4978-9c38-f85d6b5a6e55 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:09.024270 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1082/2171] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:10.231692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527fb01-4d8e-4906-bd7c-95bc185b8202 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:10.232894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1527fb01-4d8e-4906-bd7c-95bc185b8202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:10.241682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:10.263637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527fb01-4d8e-4906-bd7c-95bc185b8202 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:10.275062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:10.276933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:10.277681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:10.278223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] PATH_INFO: `/v3/projects/2c8065704a2d4030bbce3a2bf4d6caae` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:10.281166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c8065704a2d4030bbce3a2bf4d6caae)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:10.289468 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:10.483002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c8065704a2d4030bbce3a2bf4d6caae', 'request_id': 'req-bf55288b-69d6-450a-9147-4b3bf4e122e3'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:10.491078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c8065704a2d4030bbce3a2bf4d6caae', 'request_id': 'req-bf55288b-69d6-450a-9147-4b3bf4e122e3'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:10.505630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c8065704a2d4030bbce3a2bf4d6caae', 'request_id': 'req-bf55288b-69d6-450a-9147-4b3bf4e122e3'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:10.607944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] The token cache is being invalidate because project 2c8065704a2d4030bbce3a2bf4d6caae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:10.608708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:10.609712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:10.615510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1090/2172] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:10 2022] DELETE /identity/v3/projects/2c8065704a2d4030bbce3a2bf4d6caae => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:10.616631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50532e-adb2-4978-9c38-f85d6b5a6e55 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:10.629934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50532e-adb2-4978-9c38-f85d6b5a6e55 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:10.689347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:10.737862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:10.740033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:10.740033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:10.740033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] PATH_INFO: `/v3/users/7ae95f2764c64f928e71adfa196cfb62` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:10.741088 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae95f2764c64f928e71adfa196cfb62)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:10.743187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:10.949346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae95f2764c64f928e71adfa196cfb62'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:10.957931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae95f2764c64f928e71adfa196cfb62'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:11.004567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae95f2764c64f928e71adfa196cfb62'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:11.006872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae95f2764c64f928e71adfa196cfb62'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:11.009464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-052732ef-fd7f-4f75-97b5-eb44f44e666c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae95f2764c64f928e71adfa196cfb62'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:11.069723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1083/2173] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:10 2022] DELETE /identity/v3/users/7ae95f2764c64f928e71adfa196cfb62 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:11.814835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:11.816454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:11.841187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55288b-69d6-450a-9147-4b3bf4e122e3 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:11.980727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:11.982146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:11.982855 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:11.984120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] PATH_INFO: `/v3/projects/322da25f773f4d4eaf73981a98ff6a77` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:11.990584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=322da25f773f4d4eaf73981a98ff6a77)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:11.993511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:12.138456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322da25f773f4d4eaf73981a98ff6a77', 'request_id': 'req-7492af2c-5d64-4710-883e-49be29eeeba9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:12.147559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322da25f773f4d4eaf73981a98ff6a77', 'request_id': 'req-7492af2c-5d64-4710-883e-49be29eeeba9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:12.159485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322da25f773f4d4eaf73981a98ff6a77', 'request_id': 'req-7492af2c-5d64-4710-883e-49be29eeeba9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:12.267545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] The token cache is being invalidate because project 322da25f773f4d4eaf73981a98ff6a77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:12.267928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:12.268684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7492af2c-5d64-4710-883e-49be29eeeba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:12.270301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1091/2174] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:11 2022] DELETE /identity/v3/projects/322da25f773f4d4eaf73981a98ff6a77 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:12.277279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cf3ba-0cff-4cc7-8b0f-21c15e322f18 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:12.278143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cf3ba-0cff-4cc7-8b0f-21c15e322f18 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:12.278665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cf3ba-0cff-4cc7-8b0f-21c15e322f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:12.313569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e07cf3ba-0cff-4cc7-8b0f-21c15e322f18 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:12.316314 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e07cf3ba-0cff-4cc7-8b0f-21c15e322f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:12.447111 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1084/2175] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:12.455569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67d05-fa41-4762-bce8-fe5798f10023 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:12.456079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67d05-fa41-4762-bce8-fe5798f10023 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:12.456422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67d05-fa41-4762-bce8-fe5798f10023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:12.497577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8db67d05-fa41-4762-bce8-fe5798f10023 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:12.499727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8db67d05-fa41-4762-bce8-fe5798f10023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:12.510926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1092/2176] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:12.514868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cf3ba-0cff-4cc7-8b0f-21c15e322f18 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:12.532498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cf3ba-0cff-4cc7-8b0f-21c15e322f18 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:12.537331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437eaed-8305-4fd9-a593-99462946ecfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:12.539074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437eaed-8305-4fd9-a593-99462946ecfc admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:12.544653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437eaed-8305-4fd9-a593-99462946ecfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:12.544653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437eaed-8305-4fd9-a593-99462946ecfc admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:12.544653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437eaed-8305-4fd9-a593-99462946ecfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:12.544653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437eaed-8305-4fd9-a593-99462946ecfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:12.544653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437eaed-8305-4fd9-a593-99462946ecfc admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:12.566417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1085/2177] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:12.571611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67d05-fa41-4762-bce8-fe5798f10023 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:12.581787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67d05-fa41-4762-bce8-fe5798f10023 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:12.587766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdfacc-5e2b-44f7-97d4-15ac246745ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:12.588382 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdfacc-5e2b-44f7-97d4-15ac246745ef admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:12.588666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdfacc-5e2b-44f7-97d4-15ac246745ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:12.588944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdfacc-5e2b-44f7-97d4-15ac246745ef admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:12.589705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdfacc-5e2b-44f7-97d4-15ac246745ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:12.595308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdfacc-5e2b-44f7-97d4-15ac246745ef admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:12.601617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:12.717987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1093/2178] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:12 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:01:12.722740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437eaed-8305-4fd9-a593-99462946ecfc admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:12.745146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437eaed-8305-4fd9-a593-99462946ecfc admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:12.788407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:12.822433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a41214-005f-43d5-a50b-49bb135e508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:12.823558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a41214-005f-43d5-a50b-49bb135e508a admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:12.823862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a41214-005f-43d5-a50b-49bb135e508a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:12.824150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a41214-005f-43d5-a50b-49bb135e508a admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:12.824875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a41214-005f-43d5-a50b-49bb135e508a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:12.833554 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a41214-005f-43d5-a50b-49bb135e508a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:12.940924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1086/2179] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:12 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:12.943175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdfacc-5e2b-44f7-97d4-15ac246745ef admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:12.956737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdfacc-5e2b-44f7-97d4-15ac246745ef admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:12.961807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666349f1-33a1-4578-8ed1-3aa44d4af5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:12.962811 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666349f1-33a1-4578-8ed1-3aa44d4af5cc admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:12.963179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666349f1-33a1-4578-8ed1-3aa44d4af5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:12.963658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666349f1-33a1-4578-8ed1-3aa44d4af5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:12.964640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666349f1-33a1-4578-8ed1-3aa44d4af5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:12.967064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666349f1-33a1-4578-8ed1-3aa44d4af5cc admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:12.980533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1094/2180] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:01:12.982184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a41214-005f-43d5-a50b-49bb135e508a admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:12.994488 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a41214-005f-43d5-a50b-49bb135e508a admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:12.999660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608469a0-79da-43bc-aebc-f80081d80f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:13.004744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608469a0-79da-43bc-aebc-f80081d80f91 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:13.004744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608469a0-79da-43bc-aebc-f80081d80f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:13.004744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608469a0-79da-43bc-aebc-f80081d80f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:13.004744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608469a0-79da-43bc-aebc-f80081d80f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:13.004744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608469a0-79da-43bc-aebc-f80081d80f91 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:13.015891 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1087/2181] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:13.025316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666349f1-33a1-4578-8ed1-3aa44d4af5cc admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:13.036460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666349f1-33a1-4578-8ed1-3aa44d4af5cc admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:13.041960 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6bf72-5832-4794-8cc6-0c497d9944a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:13.043186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6bf72-5832-4794-8cc6-0c497d9944a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:13.043334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6bf72-5832-4794-8cc6-0c497d9944a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:13.043751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6bf72-5832-4794-8cc6-0c497d9944a2 admin admin] PATH_INFO: `/v3/projects/16538a5c07634586890491063de0ece8/users/62a66639eade4c30b994714b81a96a85/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:13.063808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6bf72-5832-4794-8cc6-0c497d9944a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16538a5c07634586890491063de0ece8, user_id=62a66639eade4c30b994714b81a96a85, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:13.069031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6bf72-5832-4794-8cc6-0c497d9944a2 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:13.154735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1095/2182] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:13 2022] PUT /identity/v3/projects/16538a5c07634586890491063de0ece8/users/62a66639eade4c30b994714b81a96a85/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:13.169370 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ed4d0-907a-4741-b1e5-f09f11779481 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:13.169720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ed4d0-907a-4741-b1e5-f09f11779481 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:13.170403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ed4d0-907a-4741-b1e5-f09f11779481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:13.212992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4a0ed4d0-907a-4741-b1e5-f09f11779481 None None] MFA Rules not processed for user `62a66639eade4c30b994714b81a96a85`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:13.214241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4a0ed4d0-907a-4741-b1e5-f09f11779481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:13.358448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1088/2183] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:13.381695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6bf72-5832-4794-8cc6-0c497d9944a2 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:13.383803 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-40b6bf72-5832-4794-8cc6-0c497d9944a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:13.462171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6bf72-5832-4794-8cc6-0c497d9944a2 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:13.470750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1235f-a04b-45c2-9412-576f1e27e411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:13.472325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1235f-a04b-45c2-9412-576f1e27e411 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:13.472613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1235f-a04b-45c2-9412-576f1e27e411 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:13.472889 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1235f-a04b-45c2-9412-576f1e27e411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:13.483006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1235f-a04b-45c2-9412-576f1e27e411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:13.484465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1235f-a04b-45c2-9412-576f1e27e411 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:13.494862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1096/2184] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:20.173512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ed4d0-907a-4741-b1e5-f09f11779481 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:20.240066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ed4d0-907a-4741-b1e5-f09f11779481 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:20.244613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:20.245481 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:20.245762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:20.246033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] PATH_INFO: `/v3/users/62a66639eade4c30b994714b81a96a85` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:20.247378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a66639eade4c30b994714b81a96a85)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:20.249286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:20.505854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:20.507748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a66639eade4c30b994714b81a96a85'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:20.517436 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a66639eade4c30b994714b81a96a85'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:20.627513 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a66639eade4c30b994714b81a96a85'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:20.628191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a66639eade4c30b994714b81a96a85'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:20.628803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a66639eade4c30b994714b81a96a85'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:20.648561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1089/2185] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:20 2022] DELETE /identity/v3/users/62a66639eade4c30b994714b81a96a85 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:20.653222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5604bf-4126-4c22-87b6-85ff67d1e450 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:20.653727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5604bf-4126-4c22-87b6-85ff67d1e450 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:20.659150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5604bf-4126-4c22-87b6-85ff67d1e450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:20.673250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-7c5604bf-4126-4c22-87b6-85ff67d1e450 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:20.674547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7c5604bf-4126-4c22-87b6-85ff67d1e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:20.741457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1097/2186] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:20.755519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:20.756943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:20.820923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dd39a-2df7-45ac-9821-de24ec098c69 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:20.824553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81561949-b137-41b4-afd8-de3d1ca96285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:20.825203 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81561949-b137-41b4-afd8-de3d1ca96285 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:20.825486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81561949-b137-41b4-afd8-de3d1ca96285 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:20.825754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81561949-b137-41b4-afd8-de3d1ca96285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:20.835544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81561949-b137-41b4-afd8-de3d1ca96285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:20.837002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81561949-b137-41b4-afd8-de3d1ca96285 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:20.846902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1090/2187] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:21.316081 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5604bf-4126-4c22-87b6-85ff67d1e450 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:21.319556 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7c5604bf-4126-4c22-87b6-85ff67d1e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:21.344915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5604bf-4126-4c22-87b6-85ff67d1e450 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:21.354440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:21.355379 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:21.355662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:21.355939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] PATH_INFO: `/v3/projects/16538a5c07634586890491063de0ece8` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:21.358241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] RBAC: Authorizing `identity:delete_project(project_id=16538a5c07634586890491063de0ece8)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:21.360881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:21.523895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16538a5c07634586890491063de0ece8', 'request_id': 'req-b9340b2d-7718-4ffa-9d65-15219d97e06c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:21.524927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:21.531376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16538a5c07634586890491063de0ece8', 'request_id': 'req-b9340b2d-7718-4ffa-9d65-15219d97e06c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:21.537833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16538a5c07634586890491063de0ece8', 'request_id': 'req-b9340b2d-7718-4ffa-9d65-15219d97e06c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:21.615524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] The token cache is being invalidate because project 16538a5c07634586890491063de0ece8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:21.615796 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:21.616423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b9340b2d-7718-4ffa-9d65-15219d97e06c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:21.617978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1098/2188] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:21 2022] DELETE /identity/v3/projects/16538a5c07634586890491063de0ece8 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:21.622914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285f588-0986-4cb4-82bf-84f5d1bf0671 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:21.624671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285f588-0986-4cb4-82bf-84f5d1bf0671 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:21.624671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285f588-0986-4cb4-82bf-84f5d1bf0671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:21.641192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-6285f588-0986-4cb4-82bf-84f5d1bf0671 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:21.642423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6285f588-0986-4cb4-82bf-84f5d1bf0671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:21.701379 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1091/2189] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:21.709682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8c0b-b401-4b98-8d36-c3a546e97bce None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:21.710241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8c0b-b401-4b98-8d36-c3a546e97bce None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:21.710241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8c0b-b401-4b98-8d36-c3a546e97bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:21.729928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-69ad8c0b-b401-4b98-8d36-c3a546e97bce None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:21.735516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-69ad8c0b-b401-4b98-8d36-c3a546e97bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:21.739504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1099/2190] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:21.743617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285f588-0986-4cb4-82bf-84f5d1bf0671 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:21.753271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285f588-0986-4cb4-82bf-84f5d1bf0671 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:21.757868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03510477-30a1-4ce6-a2ca-81bad5c3659c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:21.758595 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510477-30a1-4ce6-a2ca-81bad5c3659c admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:21.758878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510477-30a1-4ce6-a2ca-81bad5c3659c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:21.759153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510477-30a1-4ce6-a2ca-81bad5c3659c admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:21.759902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03510477-30a1-4ce6-a2ca-81bad5c3659c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:21.760206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03510477-30a1-4ce6-a2ca-81bad5c3659c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:21.761795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03510477-30a1-4ce6-a2ca-81bad5c3659c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:21.773873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1092/2191] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:21.780157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad8c0b-b401-4b98-8d36-c3a546e97bce None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:21.790069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad8c0b-b401-4b98-8d36-c3a546e97bce None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:21.794933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b339997-9702-4657-9edb-ce9d98e383ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:21.795584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339997-9702-4657-9edb-ce9d98e383ac admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:21.795863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339997-9702-4657-9edb-ce9d98e383ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:21.796135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b339997-9702-4657-9edb-ce9d98e383ac admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:21.796836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b339997-9702-4657-9edb-ce9d98e383ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:21.798813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b339997-9702-4657-9edb-ce9d98e383ac admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:21.885845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1100/2192] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:21 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:21.889899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03510477-30a1-4ce6-a2ca-81bad5c3659c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:21.900741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03510477-30a1-4ce6-a2ca-81bad5c3659c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:21.949564 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:21.958854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a334eab-e8c1-48d6-ba0f-0da923b8af31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:21.959528 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a334eab-e8c1-48d6-ba0f-0da923b8af31 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:21.959812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a334eab-e8c1-48d6-ba0f-0da923b8af31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:21.960089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a334eab-e8c1-48d6-ba0f-0da923b8af31 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:21.960787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a334eab-e8c1-48d6-ba0f-0da923b8af31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:21.964458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a334eab-e8c1-48d6-ba0f-0da923b8af31 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:22.049721 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1093/2193] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:21 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:01:22.053793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b339997-9702-4657-9edb-ce9d98e383ac admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:22.063995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b339997-9702-4657-9edb-ce9d98e383ac admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:22.069096 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960dc62-027d-463e-a5ec-87a7868bbc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:22.069747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960dc62-027d-463e-a5ec-87a7868bbc8d admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:22.070027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960dc62-027d-463e-a5ec-87a7868bbc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:22.070299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960dc62-027d-463e-a5ec-87a7868bbc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:22.071048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960dc62-027d-463e-a5ec-87a7868bbc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:22.072671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960dc62-027d-463e-a5ec-87a7868bbc8d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:22.082815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1101/2194] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:22.087072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a334eab-e8c1-48d6-ba0f-0da923b8af31 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:22.096903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a334eab-e8c1-48d6-ba0f-0da923b8af31 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:22.101502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3b238-59ec-4d29-b934-9b058f2c7201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:22.102168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3b238-59ec-4d29-b934-9b058f2c7201 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:22.102452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3b238-59ec-4d29-b934-9b058f2c7201 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:22.102757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3b238-59ec-4d29-b934-9b058f2c7201 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:22.103493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3b238-59ec-4d29-b934-9b058f2c7201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:22.105117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3b238-59ec-4d29-b934-9b058f2c7201 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:22.115513 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1094/2195] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:22.119737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960dc62-027d-463e-a5ec-87a7868bbc8d admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:22.131583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960dc62-027d-463e-a5ec-87a7868bbc8d admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:22.138373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f62533-8979-4294-b95a-0b2cfe4a4750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:22.139777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f62533-8979-4294-b95a-0b2cfe4a4750 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:22.140326 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f62533-8979-4294-b95a-0b2cfe4a4750 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:22.140752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f62533-8979-4294-b95a-0b2cfe4a4750 admin admin] PATH_INFO: `/v3/projects/291b785bb9c34b128cdd59e2b5fa52d5/users/b3de3cf7e2364dbbb829a13360b62abf/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:22.162388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f62533-8979-4294-b95a-0b2cfe4a4750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=291b785bb9c34b128cdd59e2b5fa52d5, user_id=b3de3cf7e2364dbbb829a13360b62abf, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:22.165433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f62533-8979-4294-b95a-0b2cfe4a4750 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:22.261918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1102/2196] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:22 2022] PUT /identity/v3/projects/291b785bb9c34b128cdd59e2b5fa52d5/users/b3de3cf7e2364dbbb829a13360b62abf/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:22.272460 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7011c0-1a13-42c7-90cd-05ad25efc678 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:22.272758 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7011c0-1a13-42c7-90cd-05ad25efc678 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:22.273041 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7011c0-1a13-42c7-90cd-05ad25efc678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:22.298930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-ed7011c0-1a13-42c7-90cd-05ad25efc678 None None] MFA Rules not processed for user `b3de3cf7e2364dbbb829a13360b62abf`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:22.300191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ed7011c0-1a13-42c7-90cd-05ad25efc678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:22.362410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1095/2197] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:22.366932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f62533-8979-4294-b95a-0b2cfe4a4750 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:22.431901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f62533-8979-4294-b95a-0b2cfe4a4750 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:22.437040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf7698-01d4-496b-b0cb-edce26296e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:22.437696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf7698-01d4-496b-b0cb-edce26296e8a admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:22.437978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf7698-01d4-496b-b0cb-edce26296e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:22.438291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf7698-01d4-496b-b0cb-edce26296e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:22.439009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf7698-01d4-496b-b0cb-edce26296e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:22.440885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf7698-01d4-496b-b0cb-edce26296e8a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:22.545980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1103/2198] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:22 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:22.547166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7011c0-1a13-42c7-90cd-05ad25efc678 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:22.558357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7011c0-1a13-42c7-90cd-05ad25efc678 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:22.615795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864759d6-c4d8-43d5-85c0-8338a58e745f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:22.616499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864759d6-c4d8-43d5-85c0-8338a58e745f admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:22.616786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864759d6-c4d8-43d5-85c0-8338a58e745f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:22.617067 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864759d6-c4d8-43d5-85c0-8338a58e745f admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:22.617801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864759d6-c4d8-43d5-85c0-8338a58e745f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:22.619758 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864759d6-c4d8-43d5-85c0-8338a58e745f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:22.723756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1096/2199] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:22 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:01:22.727706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf7698-01d4-496b-b0cb-edce26296e8a admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:22.738045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf7698-01d4-496b-b0cb-edce26296e8a admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:22.743294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d32f-8e97-4b3f-bbf8-975c2f8688d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:22.745306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d32f-8e97-4b3f-bbf8-975c2f8688d9 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:22.745306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d32f-8e97-4b3f-bbf8-975c2f8688d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:22.745306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d32f-8e97-4b3f-bbf8-975c2f8688d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:22.745306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692d32f-8e97-4b3f-bbf8-975c2f8688d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:22.746947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692d32f-8e97-4b3f-bbf8-975c2f8688d9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:22.758693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1104/2200] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:22.762495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864759d6-c4d8-43d5-85c0-8338a58e745f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:22.772556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864759d6-c4d8-43d5-85c0-8338a58e745f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:22.777288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bd15e-61d0-40b4-930d-919784206354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:22.777945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bd15e-61d0-40b4-930d-919784206354 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:22.778230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bd15e-61d0-40b4-930d-919784206354 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:22.778536 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bd15e-61d0-40b4-930d-919784206354 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:22.779269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bd15e-61d0-40b4-930d-919784206354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:22.780917 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bd15e-61d0-40b4-930d-919784206354 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:22.791120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1097/2201] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:22.795073 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d32f-8e97-4b3f-bbf8-975c2f8688d9 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:22.805411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d32f-8e97-4b3f-bbf8-975c2f8688d9 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:22.810181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e14c0-ac8c-4902-8c81-f05b96dcdef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:22.811034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e14c0-ac8c-4902-8c81-f05b96dcdef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:22.811315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e14c0-ac8c-4902-8c81-f05b96dcdef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:22.811589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e14c0-ac8c-4902-8c81-f05b96dcdef2 admin admin] PATH_INFO: `/v3/projects/1c95682563734148883e14ed742dea99/users/34dcce8a1d5d4555b67df6b21e0f2121/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:22.826426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e14c0-ac8c-4902-8c81-f05b96dcdef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c95682563734148883e14ed742dea99, user_id=34dcce8a1d5d4555b67df6b21e0f2121, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:22.829066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e14c0-ac8c-4902-8c81-f05b96dcdef2 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:22.889407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1105/2202] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:22 2022] PUT /identity/v3/projects/1c95682563734148883e14ed742dea99/users/34dcce8a1d5d4555b67df6b21e0f2121/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:22.899837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632e5ef-e5d6-46f8-8a86-95474cdc6c8e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:22.900141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632e5ef-e5d6-46f8-8a86-95474cdc6c8e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:22.900389 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632e5ef-e5d6-46f8-8a86-95474cdc6c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:22.927656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-8632e5ef-e5d6-46f8-8a86-95474cdc6c8e None None] MFA Rules not processed for user `34dcce8a1d5d4555b67df6b21e0f2121`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:22.928845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8632e5ef-e5d6-46f8-8a86-95474cdc6c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:22.959808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:22.994959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1098/2203] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:22.999448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e14c0-ac8c-4902-8c81-f05b96dcdef2 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:23.025777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:23.061031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e14c0-ac8c-4902-8c81-f05b96dcdef2 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:23.065573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b599bd7-1a2b-4116-a4e9-2f86d6bf555a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:23.066228 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b599bd7-1a2b-4116-a4e9-2f86d6bf555a admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.066534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b599bd7-1a2b-4116-a4e9-2f86d6bf555a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.066812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b599bd7-1a2b-4116-a4e9-2f86d6bf555a admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.067529 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b599bd7-1a2b-4116-a4e9-2f86d6bf555a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:23.069449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b599bd7-1a2b-4116-a4e9-2f86d6bf555a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:23.147900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1106/2204] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:22 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:01:23.152062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632e5ef-e5d6-46f8-8a86-95474cdc6c8e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:23.164142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632e5ef-e5d6-46f8-8a86-95474cdc6c8e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:23.228017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b8da7-f219-49fd-bee4-cbf177c416ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:23.228017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8da7-f219-49fd-bee4-cbf177c416ce admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.228017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8da7-f219-49fd-bee4-cbf177c416ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.228017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8da7-f219-49fd-bee4-cbf177c416ce admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.228418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b8da7-f219-49fd-bee4-cbf177c416ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:23.228418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b8da7-f219-49fd-bee4-cbf177c416ce admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:23.314322 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1099/2205] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:23 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:23.322142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b599bd7-1a2b-4116-a4e9-2f86d6bf555a admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:23.333762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b599bd7-1a2b-4116-a4e9-2f86d6bf555a admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:23.338646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f910e-a0cf-4d57-a481-be6fa1eb22ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:23.339321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f910e-a0cf-4d57-a481-be6fa1eb22ed admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.339686 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f910e-a0cf-4d57-a481-be6fa1eb22ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.339916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f910e-a0cf-4d57-a481-be6fa1eb22ed admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.340652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f910e-a0cf-4d57-a481-be6fa1eb22ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:23.342246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f910e-a0cf-4d57-a481-be6fa1eb22ed admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:23.357567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1107/2206] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:23.361865 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b8da7-f219-49fd-bee4-cbf177c416ce admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:23.373403 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b8da7-f219-49fd-bee4-cbf177c416ce admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:23.378572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4e141-6f26-4210-b147-d94410a6fce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:23.379496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4e141-6f26-4210-b147-d94410a6fce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.379781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4e141-6f26-4210-b147-d94410a6fce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.380204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4e141-6f26-4210-b147-d94410a6fce1 admin admin] PATH_INFO: `/v3/projects/fa1bdcc431d54c0cbcf468cd856cf0fe/users/686c579727774d84b62614e5641e2977/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.395348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4e141-6f26-4210-b147-d94410a6fce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1bdcc431d54c0cbcf468cd856cf0fe, user_id=686c579727774d84b62614e5641e2977, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:23.398603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4e141-6f26-4210-b147-d94410a6fce1 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:23.467046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1100/2207] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:23 2022] PUT /identity/v3/projects/fa1bdcc431d54c0cbcf468cd856cf0fe/users/686c579727774d84b62614e5641e2977/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:23.471080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f910e-a0cf-4d57-a481-be6fa1eb22ed admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:23.482192 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f910e-a0cf-4d57-a481-be6fa1eb22ed admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:23.537213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04915c1-4d9a-44ef-8d4a-8c650787dc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:23.538125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04915c1-4d9a-44ef-8d4a-8c650787dc68 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.538420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04915c1-4d9a-44ef-8d4a-8c650787dc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.538728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04915c1-4d9a-44ef-8d4a-8c650787dc68 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.539450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04915c1-4d9a-44ef-8d4a-8c650787dc68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:23.541035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04915c1-4d9a-44ef-8d4a-8c650787dc68 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:23.550816 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1108/2208] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:23.554927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4e141-6f26-4210-b147-d94410a6fce1 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:23.565174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4e141-6f26-4210-b147-d94410a6fce1 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:23.570223 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f81716-113e-4a34-afdf-001ab366d4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:23.571130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f81716-113e-4a34-afdf-001ab366d4fd admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.571413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f81716-113e-4a34-afdf-001ab366d4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.571685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f81716-113e-4a34-afdf-001ab366d4fd admin admin] PATH_INFO: `/v3/projects/fa1bdcc431d54c0cbcf468cd856cf0fe/users/686c579727774d84b62614e5641e2977/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.573647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f81716-113e-4a34-afdf-001ab366d4fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1bdcc431d54c0cbcf468cd856cf0fe, user_id=686c579727774d84b62614e5641e2977, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:23.576317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f81716-113e-4a34-afdf-001ab366d4fd admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:23.586916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1101/2209] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:23 2022] PUT /identity/v3/projects/fa1bdcc431d54c0cbcf468cd856cf0fe/users/686c579727774d84b62614e5641e2977/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:23.655086 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9c556-131d-405e-a486-ea7abfe62e5a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.655395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9c556-131d-405e-a486-ea7abfe62e5a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.655678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9c556-131d-405e-a486-ea7abfe62e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.682607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-0cd9c556-131d-405e-a486-ea7abfe62e5a None None] MFA Rules not processed for user `686c579727774d84b62614e5641e2977`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:23.683870 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0cd9c556-131d-405e-a486-ea7abfe62e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:23.758075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1109/2210] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:23.765785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f81716-113e-4a34-afdf-001ab366d4fd admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:23.767437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-54f81716-113e-4a34-afdf-001ab366d4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:23.824674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f81716-113e-4a34-afdf-001ab366d4fd admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:23.828210 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48823f6-c931-4e02-be7d-c9d36d2bbbcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:23.828847 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48823f6-c931-4e02-be7d-c9d36d2bbbcd service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.829131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48823f6-c931-4e02-be7d-c9d36d2bbbcd service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.829408 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48823f6-c931-4e02-be7d-c9d36d2bbbcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.862064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:23.884839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48823f6-c931-4e02-be7d-c9d36d2bbbcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:23.886286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48823f6-c931-4e02-be7d-c9d36d2bbbcd service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:23.895487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1102/2211] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:25.887987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9c556-131d-405e-a486-ea7abfe62e5a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:25.889480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0cd9c556-131d-405e-a486-ea7abfe62e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:25.905750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9c556-131d-405e-a486-ea7abfe62e5a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:25.913295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057616eb-7196-418b-b453-5d3df9cb95dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:25.913958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057616eb-7196-418b-b453-5d3df9cb95dc service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:25.915691 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057616eb-7196-418b-b453-5d3df9cb95dc service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:25.916390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057616eb-7196-418b-b453-5d3df9cb95dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:25.937488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:25.975548 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057616eb-7196-418b-b453-5d3df9cb95dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:25.976993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057616eb-7196-418b-b453-5d3df9cb95dc service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:25.986386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1110/2212] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:29.320464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48823f6-c931-4e02-be7d-c9d36d2bbbcd service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:29.329096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48823f6-c931-4e02-be7d-c9d36d2bbbcd service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:29.333072 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f48050-dc9d-4111-9e1d-070a0cb6901a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:29.333718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f48050-dc9d-4111-9e1d-070a0cb6901a service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:29.334001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f48050-dc9d-4111-9e1d-070a0cb6901a service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:29.334279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f48050-dc9d-4111-9e1d-070a0cb6901a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:29.344170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f48050-dc9d-4111-9e1d-070a0cb6901a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:29.345597 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f48050-dc9d-4111-9e1d-070a0cb6901a service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:29.355640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1103/2213] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:33.636940 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e5922-41c8-48d8-af95-c0864dffa923 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:33.637439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e5922-41c8-48d8-af95-c0864dffa923 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:33.637700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e5922-41c8-48d8-af95-c0864dffa923 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:33.639471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1111/2214] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 06:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 06:01:33.647493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067259d-6342-4d5b-806b-0240448fefb7 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:33.648603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067259d-6342-4d5b-806b-0240448fefb7 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:33.648603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067259d-6342-4d5b-806b-0240448fefb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:33.672069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-7067259d-6342-4d5b-806b-0240448fefb7 None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:33.673398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7067259d-6342-4d5b-806b-0240448fefb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:33.763422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1104/2215] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 06:01:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:33.772673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e5922-41c8-48d8-af95-c0864dffa923 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:33.783153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-952e5922-41c8-48d8-af95-c0864dffa923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:33.845507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e5922-41c8-48d8-af95-c0864dffa923 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:33.849465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb2483-dd96-4a00-8a45-d182e6ac4ea3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e12361da5f849ee97607ca68ee6ef10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:33.850574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb2483-dd96-4a00-8a45-d182e6ac4ea3 service glance] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:33.850916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb2483-dd96-4a00-8a45-d182e6ac4ea3 service glance] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:33.851767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb2483-dd96-4a00-8a45-d182e6ac4ea3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:33.863005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb2483-dd96-4a00-8a45-d182e6ac4ea3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:33.864438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb2483-dd96-4a00-8a45-d182e6ac4ea3 service glance] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:33.876629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1112/2216] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:34.090035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7bf10-fb14-4a4b-a4e3-d9f5f187edf3 None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.090035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7bf10-fb14-4a4b-a4e3-d9f5f187edf3 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.090035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7bf10-fb14-4a4b-a4e3-d9f5f187edf3 None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.090035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1105/2217] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 06:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:01:34.093744 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06419884-cc79-4564-9ea0-d49a5bde79e5 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.094080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06419884-cc79-4564-9ea0-d49a5bde79e5 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.094394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06419884-cc79-4564-9ea0-d49a5bde79e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.113097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-06419884-cc79-4564-9ea0-d49a5bde79e5 None None] MFA Rules not processed for user `ecd61347f8874b89aada19772ec56179`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:34.118417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-06419884-cc79-4564-9ea0-d49a5bde79e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:34.168410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1113/2218] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 06:01:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:34.180978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7bf10-fb14-4a4b-a4e3-d9f5f187edf3 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:34.183014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-40d7bf10-fb14-4a4b-a4e3-d9f5f187edf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:34.197164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7bf10-fb14-4a4b-a4e3-d9f5f187edf3 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:34.200742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c3dfb-0259-474f-bbfb-6914586f0122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:34.201384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c3dfb-0259-474f-bbfb-6914586f0122 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.201667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c3dfb-0259-474f-bbfb-6914586f0122 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.201939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c3dfb-0259-474f-bbfb-6914586f0122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.209307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c3dfb-0259-474f-bbfb-6914586f0122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:34.211348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c3dfb-0259-474f-bbfb-6914586f0122 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:34.219759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1106/2219] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:34.283418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06419884-cc79-4564-9ea0-d49a5bde79e5 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:34.340019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247d0eb-41e5-411f-b88a-ac6adad6b9fe None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.340435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247d0eb-41e5-411f-b88a-ac6adad6b9fe None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.340950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247d0eb-41e5-411f-b88a-ac6adad6b9fe None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.342445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1107/2220] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 06:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 06:01:34.347686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7c34f-95e0-4423-babd-03de086a9405 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.347978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7c34f-95e0-4423-babd-03de086a9405 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.348253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7c34f-95e0-4423-babd-03de086a9405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.349667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06419884-cc79-4564-9ea0-d49a5bde79e5 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:34.354909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:34.355906 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.362937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.362937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] PATH_INFO: `/v3/users/b3de3cf7e2364dbbb829a13360b62abf` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.362937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3de3cf7e2364dbbb829a13360b62abf)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:34.362937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:34.373348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:34.380715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-c2e7c34f-95e0-4423-babd-03de086a9405 None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:34.382036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c2e7c34f-95e0-4423-babd-03de086a9405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:34.392712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1108/2221] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 06:01:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:34.403133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7c34f-95e0-4423-babd-03de086a9405 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:34.412311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7c34f-95e0-4423-babd-03de086a9405 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:34.416281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25e9fb-a9a6-4ba1-bc32-fe155373f692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:34.416906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25e9fb-a9a6-4ba1-bc32-fe155373f692 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.417186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25e9fb-a9a6-4ba1-bc32-fe155373f692 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.417458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25e9fb-a9a6-4ba1-bc32-fe155373f692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.427486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25e9fb-a9a6-4ba1-bc32-fe155373f692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:34.428907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25e9fb-a9a6-4ba1-bc32-fe155373f692 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:34.446654 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1109/2222] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:01:34.546188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3de3cf7e2364dbbb829a13360b62abf'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.562955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3de3cf7e2364dbbb829a13360b62abf'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.563720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3de3cf7e2364dbbb829a13360b62abf'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.620526 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3de3cf7e2364dbbb829a13360b62abf'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.621310 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3de3cf7e2364dbbb829a13360b62abf'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.644386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1114/2223] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:34 2022] DELETE /identity/v3/users/b3de3cf7e2364dbbb829a13360b62abf => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:34.649305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85c72-1152-4d95-943a-8db5934b050a None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.649748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85c72-1152-4d95-943a-8db5934b050a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.649860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85c72-1152-4d95-943a-8db5934b050a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.667720 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-b0a85c72-1152-4d95-943a-8db5934b050a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:34.668948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b0a85c72-1152-4d95-943a-8db5934b050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:34.727018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1110/2224] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:34.737159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:34.738590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:34.797454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d389e0f-a87d-4bea-b986-16ed985f1b66 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:34.802569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158ea3-32c3-4c36-8ce9-46a67f040f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:34.805104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158ea3-32c3-4c36-8ce9-46a67f040f93 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.805104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158ea3-32c3-4c36-8ce9-46a67f040f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.805104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158ea3-32c3-4c36-8ce9-46a67f040f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.815560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158ea3-32c3-4c36-8ce9-46a67f040f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:34.817926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158ea3-32c3-4c36-8ce9-46a67f040f93 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:34.840776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1115/2225] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:35.466309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85c72-1152-4d95-943a-8db5934b050a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:35.468446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b0a85c72-1152-4d95-943a-8db5934b050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:35.482672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85c72-1152-4d95-943a-8db5934b050a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:35.489483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:35.490371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:35.490766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:35.490955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] PATH_INFO: `/v3/projects/291b785bb9c34b128cdd59e2b5fa52d5` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:35.503241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=291b785bb9c34b128cdd59e2b5fa52d5)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:35.505847 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:35.544599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:35.677830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '291b785bb9c34b128cdd59e2b5fa52d5', 'request_id': 'req-f1f84de1-a4ae-46e7-83f6-09869d6018c3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:35.696583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '291b785bb9c34b128cdd59e2b5fa52d5', 'request_id': 'req-f1f84de1-a4ae-46e7-83f6-09869d6018c3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:35.696583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '291b785bb9c34b128cdd59e2b5fa52d5', 'request_id': 'req-f1f84de1-a4ae-46e7-83f6-09869d6018c3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:35.827443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] The token cache is being invalidate because project 291b785bb9c34b128cdd59e2b5fa52d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:35.827659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:35.828274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:35.829885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1111/2226] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:35 2022] DELETE /identity/v3/projects/291b785bb9c34b128cdd59e2b5fa52d5 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:35.833959 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158ea3-32c3-4c36-8ce9-46a67f040f93 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:35.844227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158ea3-32c3-4c36-8ce9-46a67f040f93 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:35.908636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:35.909941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:35.910325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:35.911091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] PATH_INFO: `/v3/users/34dcce8a1d5d4555b67df6b21e0f2121` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:35.916198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=34dcce8a1d5d4555b67df6b21e0f2121)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:35.916198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.180857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:36.193372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34dcce8a1d5d4555b67df6b21e0f2121'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:36.201217 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34dcce8a1d5d4555b67df6b21e0f2121'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:36.201993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34dcce8a1d5d4555b67df6b21e0f2121'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:36.282235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34dcce8a1d5d4555b67df6b21e0f2121'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:36.289363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34dcce8a1d5d4555b67df6b21e0f2121'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:36.320679 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1116/2227] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:35 2022] DELETE /identity/v3/users/34dcce8a1d5d4555b67df6b21e0f2121 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:36.903823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:36.906014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:36.921626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f84de1-a4ae-46e7-83f6-09869d6018c3 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:36.982711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:36.983946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.984417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.984967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] PATH_INFO: `/v3/projects/1c95682563734148883e14ed742dea99` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.986282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c95682563734148883e14ed742dea99)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:36.988329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:37.105940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c95682563734148883e14ed742dea99', 'request_id': 'req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:37.112061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c95682563734148883e14ed742dea99', 'request_id': 'req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:37.117760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c95682563734148883e14ed742dea99', 'request_id': 'req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:37.191604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] The token cache is being invalidate because project 1c95682563734148883e14ed742dea99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:37.191856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:37.192465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:37.194126 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1112/2228] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:36 2022] DELETE /identity/v3/projects/1c95682563734148883e14ed742dea99 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:37.198610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:37.200027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:37.213386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c1f41-9d57-4ba0-8b77-65a390977bfd admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:37.268996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:37.269872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:37.270152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:37.270425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] PATH_INFO: `/v3/users/686c579727774d84b62614e5641e2977` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:37.271655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=686c579727774d84b62614e5641e2977)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:37.274617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:37.456985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '686c579727774d84b62614e5641e2977'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:37.464858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '686c579727774d84b62614e5641e2977'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:37.465927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '686c579727774d84b62614e5641e2977'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:37.556811 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '686c579727774d84b62614e5641e2977'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:37.557687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37e7df44-f877-482e-8385-22b1e8029b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '686c579727774d84b62614e5641e2977'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:37.582986 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1117/2229] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:37 2022] DELETE /identity/v3/users/686c579727774d84b62614e5641e2977 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:38.155477 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:38.157054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:38.171658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5dbb0e-0533-4269-add0-e3ce57d47ee3 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:38.206649 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:38.232150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:38.233704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:38.233704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:38.233704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] PATH_INFO: `/v3/projects/fa1bdcc431d54c0cbcf468cd856cf0fe` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:38.235108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa1bdcc431d54c0cbcf468cd856cf0fe)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:38.237382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:38.345742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa1bdcc431d54c0cbcf468cd856cf0fe', 'request_id': 'req-a9da017a-9590-468c-8fc1-344baa7f00cb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:38.352914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa1bdcc431d54c0cbcf468cd856cf0fe', 'request_id': 'req-a9da017a-9590-468c-8fc1-344baa7f00cb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:38.360312 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa1bdcc431d54c0cbcf468cd856cf0fe', 'request_id': 'req-a9da017a-9590-468c-8fc1-344baa7f00cb'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:38.443607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] The token cache is being invalidate because project fa1bdcc431d54c0cbcf468cd856cf0fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:38.443992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:38.444605 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a9da017a-9590-468c-8fc1-344baa7f00cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:38.446345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1113/2230] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:38 2022] DELETE /identity/v3/projects/fa1bdcc431d54c0cbcf468cd856cf0fe => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:38.451433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c68b7-f303-4286-9040-7cd9bdec0dcb None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:38.451903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c68b7-f303-4286-9040-7cd9bdec0dcb None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:38.452019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c68b7-f303-4286-9040-7cd9bdec0dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:38.470902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ba6c68b7-f303-4286-9040-7cd9bdec0dcb None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:38.472111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ba6c68b7-f303-4286-9040-7cd9bdec0dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:38.498942 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:38.533638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1118/2231] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:38.541853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37290-014f-4cd3-b0df-052655ec20ce None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:38.542171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37290-014f-4cd3-b0df-052655ec20ce None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:38.542823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e37290-014f-4cd3-b0df-052655ec20ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:38.563236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-25e37290-014f-4cd3-b0df-052655ec20ce None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:38.564447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-25e37290-014f-4cd3-b0df-052655ec20ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:38.574640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1114/2232] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:38.577216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c68b7-f303-4286-9040-7cd9bdec0dcb None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:38.587538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c68b7-f303-4286-9040-7cd9bdec0dcb None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:38.592101 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663c9af-389c-4edd-9e0c-96a129eec4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:38.592773 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663c9af-389c-4edd-9e0c-96a129eec4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:38.593039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663c9af-389c-4edd-9e0c-96a129eec4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:38.593342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663c9af-389c-4edd-9e0c-96a129eec4f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:38.594063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663c9af-389c-4edd-9e0c-96a129eec4f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:38.594366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663c9af-389c-4edd-9e0c-96a129eec4f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:38.595980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663c9af-389c-4edd-9e0c-96a129eec4f4 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:38.607368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1119/2233] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:38.611365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e37290-014f-4cd3-b0df-052655ec20ce None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:38.620555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e37290-014f-4cd3-b0df-052655ec20ce None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:38.625259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d62ae-b26c-49ea-b586-d09409efc5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:38.626026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d62ae-b26c-49ea-b586-d09409efc5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:38.626314 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d62ae-b26c-49ea-b586-d09409efc5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:38.626671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d62ae-b26c-49ea-b586-d09409efc5a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:38.627423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d62ae-b26c-49ea-b586-d09409efc5a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:38.629329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d62ae-b26c-49ea-b586-d09409efc5a1 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:38.792144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1115/2234] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:38 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:38.793352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663c9af-389c-4edd-9e0c-96a129eec4f4 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:38.803886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663c9af-389c-4edd-9e0c-96a129eec4f4 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:38.864735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc9fc-6055-4ae7-bb50-ff4dc8742644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:38.866429 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc9fc-6055-4ae7-bb50-ff4dc8742644 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:38.866819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc9fc-6055-4ae7-bb50-ff4dc8742644 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:38.867176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc9fc-6055-4ae7-bb50-ff4dc8742644 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:38.868724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bc9fc-6055-4ae7-bb50-ff4dc8742644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:38.870785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bc9fc-6055-4ae7-bb50-ff4dc8742644 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:38.943178 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1120/2235] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:38 2022] POST /identity/v3/users => generated 282 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:01:38.948187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d62ae-b26c-49ea-b586-d09409efc5a1 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:38.955457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d62ae-b26c-49ea-b586-d09409efc5a1 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:38.960059 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fad970-46fc-4d80-8146-9420174e11c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:38.960723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fad970-46fc-4d80-8146-9420174e11c7 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:38.961039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fad970-46fc-4d80-8146-9420174e11c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:38.961323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fad970-46fc-4d80-8146-9420174e11c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:38.962055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fad970-46fc-4d80-8146-9420174e11c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:38.963672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fad970-46fc-4d80-8146-9420174e11c7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:38.974466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1116/2236] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:38.985484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc9fc-6055-4ae7-bb50-ff4dc8742644 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:38.996383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc9fc-6055-4ae7-bb50-ff4dc8742644 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:39.001189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adba7d6-b6c7-4041-9ba5-bce7d5e73d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:39.001853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adba7d6-b6c7-4041-9ba5-bce7d5e73d37 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:39.002139 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adba7d6-b6c7-4041-9ba5-bce7d5e73d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:39.002412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adba7d6-b6c7-4041-9ba5-bce7d5e73d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:39.003161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adba7d6-b6c7-4041-9ba5-bce7d5e73d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:39.004839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adba7d6-b6c7-4041-9ba5-bce7d5e73d37 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:39.016921 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1121/2237] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:39.021356 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fad970-46fc-4d80-8146-9420174e11c7 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:39.032193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fad970-46fc-4d80-8146-9420174e11c7 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:39.037035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462960b6-9b47-4267-8b9d-40bf95ba46f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:39.037948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462960b6-9b47-4267-8b9d-40bf95ba46f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:39.038672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462960b6-9b47-4267-8b9d-40bf95ba46f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:39.038950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462960b6-9b47-4267-8b9d-40bf95ba46f2 admin admin] PATH_INFO: `/v3/projects/7e73d44fd476464eab1a9b126ece22e7/users/28d24219078e4e8e8b58299c82968844/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:39.057220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462960b6-9b47-4267-8b9d-40bf95ba46f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e73d44fd476464eab1a9b126ece22e7, user_id=28d24219078e4e8e8b58299c82968844, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:39.059944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462960b6-9b47-4267-8b9d-40bf95ba46f2 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:39.142529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1117/2238] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:39 2022] PUT /identity/v3/projects/7e73d44fd476464eab1a9b126ece22e7/users/28d24219078e4e8e8b58299c82968844/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:39.152983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd1dfd-8696-4f77-a6d9-2b25e621b1f9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:39.153276 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd1dfd-8696-4f77-a6d9-2b25e621b1f9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:39.153561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd1dfd-8696-4f77-a6d9-2b25e621b1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:39.182138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-88cd1dfd-8696-4f77-a6d9-2b25e621b1f9 None None] MFA Rules not processed for user `28d24219078e4e8e8b58299c82968844`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:39.183596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-88cd1dfd-8696-4f77-a6d9-2b25e621b1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:39.251246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1122/2239] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:01:39 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:39.262238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462960b6-9b47-4267-8b9d-40bf95ba46f2 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:39.264085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-462960b6-9b47-4267-8b9d-40bf95ba46f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:39.319499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462960b6-9b47-4267-8b9d-40bf95ba46f2 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:39.323078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407ef3a-077c-4975-8e28-0810239f3c71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:39.323717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407ef3a-077c-4975-8e28-0810239f3c71 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:39.323999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407ef3a-077c-4975-8e28-0810239f3c71 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:39.324281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407ef3a-077c-4975-8e28-0810239f3c71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:39.332392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407ef3a-077c-4975-8e28-0810239f3c71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:39.333827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407ef3a-077c-4975-8e28-0810239f3c71 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:39.342412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1118/2240] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:39 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:01:41.113883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd981599-bd85-42eb-8f7f-b9607127a18b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:41.114406 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd981599-bd85-42eb-8f7f-b9607127a18b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:41.114537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd981599-bd85-42eb-8f7f-b9607127a18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:41.133951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-bd981599-bd85-42eb-8f7f-b9607127a18b None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:41.135214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bd981599-bd85-42eb-8f7f-b9607127a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:41.156827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:41.202585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1123/2241] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 06:01:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:41.221102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407ef3a-077c-4975-8e28-0810239f3c71 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:41.224091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2407ef3a-077c-4975-8e28-0810239f3c71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:41.243019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:41.308621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407ef3a-077c-4975-8e28-0810239f3c71 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:41.313644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a7ee4-c3da-40d3-874b-cf11699f92ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:41.314291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a7ee4-c3da-40d3-874b-cf11699f92ba service swift] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:41.314613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a7ee4-c3da-40d3-874b-cf11699f92ba service swift] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:41.314893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a7ee4-c3da-40d3-874b-cf11699f92ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:41.322926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a7ee4-c3da-40d3-874b-cf11699f92ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:41.324522 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a7ee4-c3da-40d3-874b-cf11699f92ba service swift] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:41.332495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1119/2242] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:01:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:01:41.934212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd981599-bd85-42eb-8f7f-b9607127a18b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:41.936235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bd981599-bd85-42eb-8f7f-b9607127a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:42.108996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd981599-bd85-42eb-8f7f-b9607127a18b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:42.122595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5aa160-f37b-41fa-a3f6-c8ccde2626eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:42.123381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5aa160-f37b-41fa-a3f6-c8ccde2626eb service placement] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:42.123668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5aa160-f37b-41fa-a3f6-c8ccde2626eb service placement] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:42.123945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5aa160-f37b-41fa-a3f6-c8ccde2626eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:42.129037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6b5aa160-f37b-41fa-a3f6-c8ccde2626eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:42.156658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5aa160-f37b-41fa-a3f6-c8ccde2626eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:42.158606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5aa160-f37b-41fa-a3f6-c8ccde2626eb service placement] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:42.186608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1124/2243] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:47.340510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9211a-4f1d-42d9-96a7-0ce6a3e3be3f None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:47.342223 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9211a-4f1d-42d9-96a7-0ce6a3e3be3f None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:47.342838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9211a-4f1d-42d9-96a7-0ce6a3e3be3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:47.369316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-cbf9211a-4f1d-42d9-96a7-0ce6a3e3be3f None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:47.371134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cbf9211a-4f1d-42d9-96a7-0ce6a3e3be3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:47.390042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1120/2244] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 06:01:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:47.404109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5aa160-f37b-41fa-a3f6-c8ccde2626eb service placement] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:47.437235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5aa160-f37b-41fa-a3f6-c8ccde2626eb service placement] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:47.446465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3038f6-a071-4b6a-b289-f03712f1cef4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:47.447552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3038f6-a071-4b6a-b289-f03712f1cef4 service swift] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:47.448056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3038f6-a071-4b6a-b289-f03712f1cef4 service swift] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:47.448553 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3038f6-a071-4b6a-b289-f03712f1cef4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:47.460663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3038f6-a071-4b6a-b289-f03712f1cef4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:47.462635 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3038f6-a071-4b6a-b289-f03712f1cef4 service swift] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:47.479892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1125/2245] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:01:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:01:57.361697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=33 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:57.471571 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=51 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:01.939696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f96a3-4c80-4640-84ef-a6646a1b5452 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:01.940768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f96a3-4c80-4640-84ef-a6646a1b5452 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:01.941296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f96a3-4c80-4640-84ef-a6646a1b5452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:01.965319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-3d6f96a3-4c80-4640-84ef-a6646a1b5452 None None] MFA Rules not processed for user `99feeb6cf0fa46b4aabecefcdf56deff`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:01.966934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3d6f96a3-4c80-4640-84ef-a6646a1b5452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:02.036939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1121/2246] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:07.602510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd46b4-46f5-46a0-9234-53d65e71cf48 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:07.603124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd46b4-46f5-46a0-9234-53d65e71cf48 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:07.603222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd46b4-46f5-46a0-9234-53d65e71cf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:07.623519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-40bd46b4-46f5-46a0-9234-53d65e71cf48 None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:07.624778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-40bd46b4-46f5-46a0-9234-53d65e71cf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:07.633005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1126/2247] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 06:02:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:07.653559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f96a3-4c80-4640-84ef-a6646a1b5452 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:07.666447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f96a3-4c80-4640-84ef-a6646a1b5452 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:07.672054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3ef8a-e112-4f0c-9d70-f5daaa054473 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:07.672932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3ef8a-e112-4f0c-9d70-f5daaa054473 service swift] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:07.673164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3ef8a-e112-4f0c-9d70-f5daaa054473 service swift] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:07.673449 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3ef8a-e112-4f0c-9d70-f5daaa054473 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:07.684875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3ef8a-e112-4f0c-9d70-f5daaa054473 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:07.689829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3ef8a-e112-4f0c-9d70-f5daaa054473 service swift] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:07.696877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1122/2248] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:02:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:02:11.069013 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd46b4-46f5-46a0-9234-53d65e71cf48 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:11.073488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-40bd46b4-46f5-46a0-9234-53d65e71cf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:11.137607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd46b4-46f5-46a0-9234-53d65e71cf48 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:11.141939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:11.142852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:11.143137 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:11.143414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] PATH_INFO: `/v3/users/a01061da6fc0404facce47e669ec80af` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:11.144613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a01061da6fc0404facce47e669ec80af)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:11.146630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:11.379828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a01061da6fc0404facce47e669ec80af'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:11.388307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a01061da6fc0404facce47e669ec80af'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:11.389099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a01061da6fc0404facce47e669ec80af'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:11.458762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a01061da6fc0404facce47e669ec80af'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:11.459378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-89bbaccb-3585-4581-9502-cb91fcf4f55f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a01061da6fc0404facce47e669ec80af'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:11.478007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1127/2249] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:02:11 2022] DELETE /identity/v3/users/a01061da6fc0404facce47e669ec80af => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:11.748325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3ef8a-e112-4f0c-9d70-f5daaa054473 service swift] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:11.759146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3ef8a-e112-4f0c-9d70-f5daaa054473 service swift] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:11.828039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:11.829494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:11.830001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:11.830689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] PATH_INFO: `/v3/projects/011cdb0f00b9444c812ac101bc5b43e3` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:11.832768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=011cdb0f00b9444c812ac101bc5b43e3)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:11.837250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:11.937801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011cdb0f00b9444c812ac101bc5b43e3', 'request_id': 'req-21e8b3f0-3f0d-4746-bda9-342871a437b3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:11.944852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011cdb0f00b9444c812ac101bc5b43e3', 'request_id': 'req-21e8b3f0-3f0d-4746-bda9-342871a437b3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:11.953526 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011cdb0f00b9444c812ac101bc5b43e3', 'request_id': 'req-21e8b3f0-3f0d-4746-bda9-342871a437b3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:12.015766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] The token cache is being invalidate because project 011cdb0f00b9444c812ac101bc5b43e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:02:12.016328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:12.016952 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-21e8b3f0-3f0d-4746-bda9-342871a437b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:12.018593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1123/2250] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:02:11 2022] DELETE /identity/v3/projects/011cdb0f00b9444c812ac101bc5b43e3 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:12.023425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8782e0-dab0-4023-91f6-b5f4dd6438c4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:12.023712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8782e0-dab0-4023-91f6-b5f4dd6438c4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:12.024045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8782e0-dab0-4023-91f6-b5f4dd6438c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:12.043405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ff8782e0-dab0-4023-91f6-b5f4dd6438c4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:12.044642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ff8782e0-dab0-4023-91f6-b5f4dd6438c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:12.108855 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1128/2251] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:02:12.116437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5676d88-ca72-4395-af2c-3e517ef0f108 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:12.116729 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5676d88-ca72-4395-af2c-3e517ef0f108 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:12.117010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5676d88-ca72-4395-af2c-3e517ef0f108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:12.137006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d5676d88-ca72-4395-af2c-3e517ef0f108 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:12.138605 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d5676d88-ca72-4395-af2c-3e517ef0f108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:12.146860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1124/2252] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:12.151267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8782e0-dab0-4023-91f6-b5f4dd6438c4 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:12.161677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8782e0-dab0-4023-91f6-b5f4dd6438c4 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:12.167803 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc627a-c6a5-4ae3-a74b-ee95054e20b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:12.168014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc627a-c6a5-4ae3-a74b-ee95054e20b3 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:12.168185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc627a-c6a5-4ae3-a74b-ee95054e20b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:12.168451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc627a-c6a5-4ae3-a74b-ee95054e20b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:12.169194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc627a-c6a5-4ae3-a74b-ee95054e20b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:02:12.169532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc627a-c6a5-4ae3-a74b-ee95054e20b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:12.171188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc627a-c6a5-4ae3-a74b-ee95054e20b3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:12.180580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:12.187619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1129/2253] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:02:12.188884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5676d88-ca72-4395-af2c-3e517ef0f108 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:12.201297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5676d88-ca72-4395-af2c-3e517ef0f108 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:12.206096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eca542-c3e4-4b95-95ff-a3d8eaa5f1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:12.206792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eca542-c3e4-4b95-95ff-a3d8eaa5f1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:12.207073 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eca542-c3e4-4b95-95ff-a3d8eaa5f1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:12.207354 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eca542-c3e4-4b95-95ff-a3d8eaa5f1a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:12.208052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eca542-c3e4-4b95-95ff-a3d8eaa5f1a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:12.210017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eca542-c3e4-4b95-95ff-a3d8eaa5f1a5 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:12.224110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:12.287733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1125/2254] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:02:12 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:02:12.292980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc627a-c6a5-4ae3-a74b-ee95054e20b3 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:12.307844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc627a-c6a5-4ae3-a74b-ee95054e20b3 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:12.365881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51711b4f-399f-42c3-943b-c8e8ab9aa69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:12.366930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51711b4f-399f-42c3-943b-c8e8ab9aa69f admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:12.367214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51711b4f-399f-42c3-943b-c8e8ab9aa69f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:12.367494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51711b4f-399f-42c3-943b-c8e8ab9aa69f admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:12.370593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51711b4f-399f-42c3-943b-c8e8ab9aa69f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:12.372016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51711b4f-399f-42c3-943b-c8e8ab9aa69f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:12.487288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1130/2255] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:02:12 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:02:12.488791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eca542-c3e4-4b95-95ff-a3d8eaa5f1a5 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:12.500219 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eca542-c3e4-4b95-95ff-a3d8eaa5f1a5 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:12.504932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91029f-aa2b-45b9-9c49-fb3738bd1193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:12.505603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91029f-aa2b-45b9-9c49-fb3738bd1193 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:12.505890 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91029f-aa2b-45b9-9c49-fb3738bd1193 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:12.506167 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91029f-aa2b-45b9-9c49-fb3738bd1193 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:12.506942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91029f-aa2b-45b9-9c49-fb3738bd1193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:12.509192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91029f-aa2b-45b9-9c49-fb3738bd1193 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:12.520018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1126/2256] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:02:12.524923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51711b4f-399f-42c3-943b-c8e8ab9aa69f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:12.540798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51711b4f-399f-42c3-943b-c8e8ab9aa69f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:12.545038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85a0cf-9a14-4345-940f-923d321ce94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:12.545707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85a0cf-9a14-4345-940f-923d321ce94b admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:12.545996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85a0cf-9a14-4345-940f-923d321ce94b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:12.546273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85a0cf-9a14-4345-940f-923d321ce94b admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:12.547047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85a0cf-9a14-4345-940f-923d321ce94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:12.548642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85a0cf-9a14-4345-940f-923d321ce94b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:12.559724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1131/2257] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:02:12.564299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91029f-aa2b-45b9-9c49-fb3738bd1193 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:12.574927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91029f-aa2b-45b9-9c49-fb3738bd1193 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:12.579898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cd8f5-682f-4296-87b4-a9d35bf96583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:12.580821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cd8f5-682f-4296-87b4-a9d35bf96583 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:12.581123 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cd8f5-682f-4296-87b4-a9d35bf96583 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:12.581399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cd8f5-682f-4296-87b4-a9d35bf96583 admin admin] PATH_INFO: `/v3/projects/32545e8f4fe54c899a283225c9745e4b/users/602cc406ea5f4d669aaf5f28666186fe/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:12.596841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586cd8f5-682f-4296-87b4-a9d35bf96583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32545e8f4fe54c899a283225c9745e4b, user_id=602cc406ea5f4d669aaf5f28666186fe, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:12.599573 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586cd8f5-682f-4296-87b4-a9d35bf96583 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:12.672096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1127/2258] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:02:12 2022] PUT /identity/v3/projects/32545e8f4fe54c899a283225c9745e4b/users/602cc406ea5f4d669aaf5f28666186fe/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:02:12.683398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dfe5c-5376-4719-a24c-0f5baf11fc66 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:12.683688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dfe5c-5376-4719-a24c-0f5baf11fc66 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:12.683970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dfe5c-5376-4719-a24c-0f5baf11fc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:12.713450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-829dfe5c-5376-4719-a24c-0f5baf11fc66 None None] MFA Rules not processed for user `602cc406ea5f4d669aaf5f28666186fe`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:12.716277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-829dfe5c-5376-4719-a24c-0f5baf11fc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:12.800712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1132/2259] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:12.813020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cd8f5-682f-4296-87b4-a9d35bf96583 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:12.814791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-586cd8f5-682f-4296-87b4-a9d35bf96583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:12.877173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cd8f5-682f-4296-87b4-a9d35bf96583 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:12.880809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1664e-b19d-4240-bab3-480bb4d0e7de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:12.881456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1664e-b19d-4240-bab3-480bb4d0e7de service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:12.881738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1664e-b19d-4240-bab3-480bb4d0e7de service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:12.882014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1664e-b19d-4240-bab3-480bb4d0e7de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:12.891002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1664e-b19d-4240-bab3-480bb4d0e7de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:12.892486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1664e-b19d-4240-bab3-480bb4d0e7de service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:12.902257 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1128/2260] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:02:21.043409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdaa6e-76d2-4b98-af00-a0c4ba34ee7b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:21.043884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdaa6e-76d2-4b98-af00-a0c4ba34ee7b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:21.043977 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdaa6e-76d2-4b98-af00-a0c4ba34ee7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:21.063016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-48cdaa6e-76d2-4b98-af00-a0c4ba34ee7b None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:21.064313 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-48cdaa6e-76d2-4b98-af00-a0c4ba34ee7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:21.118196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1133/2261] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 06:02:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:21.130996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1664e-b19d-4240-bab3-480bb4d0e7de service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:21.132466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-dcf1664e-b19d-4240-bab3-480bb4d0e7de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:21.201635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1664e-b19d-4240-bab3-480bb4d0e7de service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:21.206736 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621987f-18da-4d3d-b6e5-aef06106dbda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:21.207387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621987f-18da-4d3d-b6e5-aef06106dbda service swift] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:21.207672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621987f-18da-4d3d-b6e5-aef06106dbda service swift] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:21.207958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621987f-18da-4d3d-b6e5-aef06106dbda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:21.215906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621987f-18da-4d3d-b6e5-aef06106dbda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:21.217508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621987f-18da-4d3d-b6e5-aef06106dbda service swift] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:21.225496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1129/2262] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:02:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:02:31.111417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:31.222282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:INSERT=2,keystone:SELECT=75 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:33.096972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdaa6e-76d2-4b98-af00-a0c4ba34ee7b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:33.099170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-48cdaa6e-76d2-4b98-af00-a0c4ba34ee7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:33.183374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdaa6e-76d2-4b98-af00-a0c4ba34ee7b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:33.187038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5ba44-6490-4c40-99ee-3b036ac0ad5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:33.187698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5ba44-6490-4c40-99ee-3b036ac0ad5c service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:33.187985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5ba44-6490-4c40-99ee-3b036ac0ad5c service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:33.188398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5ba44-6490-4c40-99ee-3b036ac0ad5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:33.190198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-41e5ba44-6490-4c40-99ee-3b036ac0ad5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:33.261640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5ba44-6490-4c40-99ee-3b036ac0ad5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:33.263721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5ba44-6490-4c40-99ee-3b036ac0ad5c service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:33.274054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1134/2263] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:02:34.210089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621987f-18da-4d3d-b6e5-aef06106dbda service swift] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:34.211828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4621987f-18da-4d3d-b6e5-aef06106dbda service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:34.286753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621987f-18da-4d3d-b6e5-aef06106dbda service swift] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:34.293690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:34.294925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:34.295213 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:34.295496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] PATH_INFO: `/v3/users/28d24219078e4e8e8b58299c82968844` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:34.296718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d24219078e4e8e8b58299c82968844)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:34.298876 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:34.535767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d24219078e4e8e8b58299c82968844'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:34.551442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d24219078e4e8e8b58299c82968844'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:34.643841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d24219078e4e8e8b58299c82968844'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:34.644635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d24219078e4e8e8b58299c82968844'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:34.645249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d24219078e4e8e8b58299c82968844'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:34.670306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1130/2264] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:02:34 2022] DELETE /identity/v3/users/28d24219078e4e8e8b58299c82968844 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:34.675010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02db17-eb51-44f9-86ee-7c564dffe024 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:34.675608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02db17-eb51-44f9-86ee-7c564dffe024 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:34.676999 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02db17-eb51-44f9-86ee-7c564dffe024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:34.697934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ea02db17-eb51-44f9-86ee-7c564dffe024 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:34.700126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ea02db17-eb51-44f9-86ee-7c564dffe024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:34.790087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1135/2265] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:02:34.800508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:34.802133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:34.888777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace999d-8698-440d-8b78-cad6c51f327c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:34.897527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a4278-dd01-4d42-b16d-f224c0b5a6c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:34.898229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a4278-dd01-4d42-b16d-f224c0b5a6c4 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:34.898534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a4278-dd01-4d42-b16d-f224c0b5a6c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:34.898817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a4278-dd01-4d42-b16d-f224c0b5a6c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:34.913794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a4278-dd01-4d42-b16d-f224c0b5a6c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:34.915834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a4278-dd01-4d42-b16d-f224c0b5a6c4 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:34.928570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1131/2266] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:02:35.555786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02db17-eb51-44f9-86ee-7c564dffe024 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:35.557282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ea02db17-eb51-44f9-86ee-7c564dffe024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:35.574807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02db17-eb51-44f9-86ee-7c564dffe024 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:35.585919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:35.587819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:35.588291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:35.588743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] PATH_INFO: `/v3/projects/7e73d44fd476464eab1a9b126ece22e7` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:35.589996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e73d44fd476464eab1a9b126ece22e7)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:35.592767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:35.730396 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:35.734422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e73d44fd476464eab1a9b126ece22e7', 'request_id': 'req-bb5bb10c-fde7-45b1-bde2-6b45e8547603'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:35.741768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e73d44fd476464eab1a9b126ece22e7', 'request_id': 'req-bb5bb10c-fde7-45b1-bde2-6b45e8547603'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:35.770227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e73d44fd476464eab1a9b126ece22e7', 'request_id': 'req-bb5bb10c-fde7-45b1-bde2-6b45e8547603'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:35.867774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] The token cache is being invalidate because project 7e73d44fd476464eab1a9b126ece22e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:02:35.870644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:35.870644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb5bb10c-fde7-45b1-bde2-6b45e8547603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:35.872185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1136/2267] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:02:35 2022] DELETE /identity/v3/projects/7e73d44fd476464eab1a9b126ece22e7 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:35.877535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9939a-4349-4e3b-bd33-30b70e952284 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:35.878395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9939a-4349-4e3b-bd33-30b70e952284 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:35.878874 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9939a-4349-4e3b-bd33-30b70e952284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:35.898228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-43a9939a-4349-4e3b-bd33-30b70e952284 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:35.900053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-43a9939a-4349-4e3b-bd33-30b70e952284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:35.918697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:35.966134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1132/2268] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:35.974896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036f79e-ce09-41fd-b29a-b1772ce72d18 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:35.975191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036f79e-ce09-41fd-b29a-b1772ce72d18 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:35.975470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036f79e-ce09-41fd-b29a-b1772ce72d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:35.995878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-d036f79e-ce09-41fd-b29a-b1772ce72d18 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:35.997175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d036f79e-ce09-41fd-b29a-b1772ce72d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:36.006307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1137/2269] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:36.010804 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9939a-4349-4e3b-bd33-30b70e952284 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:36.024189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9939a-4349-4e3b-bd33-30b70e952284 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:36.034408 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752e1d0-7de9-4034-934f-b0027a09a68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:36.038548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752e1d0-7de9-4034-934f-b0027a09a68f admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.038548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752e1d0-7de9-4034-934f-b0027a09a68f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.038548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752e1d0-7de9-4034-934f-b0027a09a68f admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.038548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752e1d0-7de9-4034-934f-b0027a09a68f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:02:36.038548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752e1d0-7de9-4034-934f-b0027a09a68f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:36.039742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752e1d0-7de9-4034-934f-b0027a09a68f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:36.054432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1133/2270] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:02:36.058742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036f79e-ce09-41fd-b29a-b1772ce72d18 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:36.064606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752e1d0-7de9-4034-934f-b0027a09a68f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:36.066837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f752e1d0-7de9-4034-934f-b0027a09a68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:36.069764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036f79e-ce09-41fd-b29a-b1772ce72d18 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:36.076186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973c00b-e362-4372-a1cf-e4fb21667f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:36.077261 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973c00b-e362-4372-a1cf-e4fb21667f99 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.077818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973c00b-e362-4372-a1cf-e4fb21667f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.078335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973c00b-e362-4372-a1cf-e4fb21667f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.079289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973c00b-e362-4372-a1cf-e4fb21667f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:36.081483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973c00b-e362-4372-a1cf-e4fb21667f99 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:36.084483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752e1d0-7de9-4034-934f-b0027a09a68f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:36.092920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:36.093879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.094684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.094684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] PATH_INFO: `/v3/users/602cc406ea5f4d669aaf5f28666186fe` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.097413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=602cc406ea5f4d669aaf5f28666186fe)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:36.099631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:36.201450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1138/2271] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:02:36 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:02:36.205396 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973c00b-e362-4372-a1cf-e4fb21667f99 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:36.280037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973c00b-e362-4372-a1cf-e4fb21667f99 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:36.285187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df7db4-4841-43b0-9294-b7fdbcd6a1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:36.292732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df7db4-4841-43b0-9294-b7fdbcd6a1ac admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.292732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df7db4-4841-43b0-9294-b7fdbcd6a1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.292732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df7db4-4841-43b0-9294-b7fdbcd6a1ac admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.292732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df7db4-4841-43b0-9294-b7fdbcd6a1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:36.292732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df7db4-4841-43b0-9294-b7fdbcd6a1ac admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:36.328098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '602cc406ea5f4d669aaf5f28666186fe'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:36.338441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '602cc406ea5f4d669aaf5f28666186fe'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:36.399149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1139/2272] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:02:36 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:02:36.403119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df7db4-4841-43b0-9294-b7fdbcd6a1ac admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:36.414720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df7db4-4841-43b0-9294-b7fdbcd6a1ac admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:36.420356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dd0e4-7a4a-47df-ba2f-2c60024b1a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:36.424705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dd0e4-7a4a-47df-ba2f-2c60024b1a92 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.424705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dd0e4-7a4a-47df-ba2f-2c60024b1a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.424949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '602cc406ea5f4d669aaf5f28666186fe'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:36.425053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dd0e4-7a4a-47df-ba2f-2c60024b1a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.425143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '602cc406ea5f4d669aaf5f28666186fe'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:36.425250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04dd0e4-7a4a-47df-ba2f-2c60024b1a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:36.425342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-93c0f498-8eba-4b8a-8984-62b65a4abdf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '602cc406ea5f4d669aaf5f28666186fe'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:36.425431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04dd0e4-7a4a-47df-ba2f-2c60024b1a92 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:36.437047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1140/2273] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:02:36.440779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dd0e4-7a4a-47df-ba2f-2c60024b1a92 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:36.442918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f04dd0e4-7a4a-47df-ba2f-2c60024b1a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:36.457035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1134/2274] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:02:36 2022] DELETE /identity/v3/users/602cc406ea5f4d669aaf5f28666186fe => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:36.458461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def89c-e5c9-4273-bf9c-7c38f3412790 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.458869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def89c-e5c9-4273-bf9c-7c38f3412790 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.459354 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def89c-e5c9-4273-bf9c-7c38f3412790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.464055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dd0e4-7a4a-47df-ba2f-2c60024b1a92 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:36.485724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-80def89c-e5c9-4273-bf9c-7c38f3412790 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:36.486981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-80def89c-e5c9-4273-bf9c-7c38f3412790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:36.535434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaffdd-1e5b-4932-bb9a-ff8a32e56dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:36.536125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaffdd-1e5b-4932-bb9a-ff8a32e56dba admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.536416 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaffdd-1e5b-4932-bb9a-ff8a32e56dba admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.536680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaffdd-1e5b-4932-bb9a-ff8a32e56dba admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.537411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedaffdd-1e5b-4932-bb9a-ff8a32e56dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:36.539490 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedaffdd-1e5b-4932-bb9a-ff8a32e56dba admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:36.553787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1141/2275] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:02:36.557432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaffdd-1e5b-4932-bb9a-ff8a32e56dba admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:36.564703 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1135/2276] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:36.569588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaffdd-1e5b-4932-bb9a-ff8a32e56dba admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:36.574377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb5592-587c-47ae-83e1-7f918145538d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:36.575335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb5592-587c-47ae-83e1-7f918145538d admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.575622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb5592-587c-47ae-83e1-7f918145538d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.575929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb5592-587c-47ae-83e1-7f918145538d admin admin] PATH_INFO: `/v3/projects/30bfa024bb4a475e9f914d91f6d01a8a/users/a3e574d24a714fc4a0a2877b63829f26/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.587496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80def89c-e5c9-4273-bf9c-7c38f3412790 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:36.589559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-80def89c-e5c9-4273-bf9c-7c38f3412790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:36.598277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb5592-587c-47ae-83e1-7f918145538d admin admin] RBAC: Authorizing `identity:create_grant(project_id=30bfa024bb4a475e9f914d91f6d01a8a, user_id=a3e574d24a714fc4a0a2877b63829f26, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:36.601008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb5592-587c-47ae-83e1-7f918145538d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:36.604705 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:36.620490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:36.657082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80def89c-e5c9-4273-bf9c-7c38f3412790 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:36.662656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e6557-c569-4819-bcaa-fb8c18c53d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:36.663483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6557-c569-4819-bcaa-fb8c18c53d68 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.663872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6557-c569-4819-bcaa-fb8c18c53d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.664502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e6557-c569-4819-bcaa-fb8c18c53d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.676898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e6557-c569-4819-bcaa-fb8c18c53d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:36.678291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e6557-c569-4819-bcaa-fb8c18c53d68 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:36.687201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1142/2277] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:02:36 2022] PUT /identity/v3/projects/30bfa024bb4a475e9f914d91f6d01a8a/users/a3e574d24a714fc4a0a2877b63829f26/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:36.698410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2c047-a54a-4143-ab13-f36abf840eb7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.698737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2c047-a54a-4143-ab13-f36abf840eb7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.699021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2c047-a54a-4143-ab13-f36abf840eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.701807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1136/2278] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:02:36.730815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2cc2c047-a54a-4143-ab13-f36abf840eb7 None None] MFA Rules not processed for user `a3e574d24a714fc4a0a2877b63829f26`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:36.732606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2cc2c047-a54a-4143-ab13-f36abf840eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:36.814114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1143/2279] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:36.825126 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e6557-c569-4819-bcaa-fb8c18c53d68 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:36.827109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-335e6557-c569-4819-bcaa-fb8c18c53d68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:36.882272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e6557-c569-4819-bcaa-fb8c18c53d68 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:36.949805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3bfc5-813e-484e-8010-151de13769cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:36.951360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3bfc5-813e-484e-8010-151de13769cb service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.951773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3bfc5-813e-484e-8010-151de13769cb service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.952054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3bfc5-813e-484e-8010-151de13769cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.966247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3bfc5-813e-484e-8010-151de13769cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:36.967699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3bfc5-813e-484e-8010-151de13769cb service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:36.979174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1137/2280] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:02:37.233122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2c047-a54a-4143-ab13-f36abf840eb7 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:37.233122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2cc2c047-a54a-4143-ab13-f36abf840eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:37.299981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2c047-a54a-4143-ab13-f36abf840eb7 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:37.304505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:37.305379 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:37.305688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:37.306000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] PATH_INFO: `/v3/projects/32545e8f4fe54c899a283225c9745e4b` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:37.307213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32545e8f4fe54c899a283225c9745e4b)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:37.309139 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:37.415168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32545e8f4fe54c899a283225c9745e4b', 'request_id': 'req-13729153-5a3a-4986-84f7-5331c574b820'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:37.421252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32545e8f4fe54c899a283225c9745e4b', 'request_id': 'req-13729153-5a3a-4986-84f7-5331c574b820'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:37.427495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32545e8f4fe54c899a283225c9745e4b', 'request_id': 'req-13729153-5a3a-4986-84f7-5331c574b820'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:37.500694 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] The token cache is being invalidate because project 32545e8f4fe54c899a283225c9745e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:02:37.500991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:37.501682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-13729153-5a3a-4986-84f7-5331c574b820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:37.503312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1144/2281] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:02:37 2022] DELETE /identity/v3/projects/32545e8f4fe54c899a283225c9745e4b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:38.653854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d9abf-531f-476e-8c1b-bd12a2db3a97 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:38.654882 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d9abf-531f-476e-8c1b-bd12a2db3a97 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:38.654998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d9abf-531f-476e-8c1b-bd12a2db3a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:38.673556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-917d9abf-531f-476e-8c1b-bd12a2db3a97 None None] MFA Rules not processed for user `a3e574d24a714fc4a0a2877b63829f26`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:38.674796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-917d9abf-531f-476e-8c1b-bd12a2db3a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:38.727749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1138/2282] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:47.496823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:48.721239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=87 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:58.461849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22339-b0f6-4c85-8b92-f03f8fb69545 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:58.462340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22339-b0f6-4c85-8b92-f03f8fb69545 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:58.462453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22339-b0f6-4c85-8b92-f03f8fb69545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:58.480367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a7c22339-b0f6-4c85-8b92-f03f8fb69545 None None] MFA Rules not processed for user `6dc49525e3394668b4f1610c4badc144`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:58.481586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a7c22339-b0f6-4c85-8b92-f03f8fb69545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:58.537018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1145/2283] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:02:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:05.696543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2358f-6645-450f-8a31-5eee03f846c9 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:05.698711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2358f-6645-450f-8a31-5eee03f846c9 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:05.699183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2358f-6645-450f-8a31-5eee03f846c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:05.727060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d3b2358f-6645-450f-8a31-5eee03f846c9 None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:05.728400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d3b2358f-6645-450f-8a31-5eee03f846c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:05.794244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1139/2284] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 06:03:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:05.817501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c22339-b0f6-4c85-8b92-f03f8fb69545 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:05.819542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a7c22339-b0f6-4c85-8b92-f03f8fb69545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:05.910132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c22339-b0f6-4c85-8b92-f03f8fb69545 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:05.917334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51209b3c-2ac0-4bc3-98e9-0f9c4cd9be10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:05.918402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51209b3c-2ac0-4bc3-98e9-0f9c4cd9be10 service swift] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:05.918402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51209b3c-2ac0-4bc3-98e9-0f9c4cd9be10 service swift] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:05.918402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51209b3c-2ac0-4bc3-98e9-0f9c4cd9be10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:05.939298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51209b3c-2ac0-4bc3-98e9-0f9c4cd9be10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:05.940714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51209b3c-2ac0-4bc3-98e9-0f9c4cd9be10 service swift] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:05.960467 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1146/2285] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:03:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:03:06.879902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59725c43-85e0-4633-b9c0-8e6b28ebaba5 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:06.880452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59725c43-85e0-4633-b9c0-8e6b28ebaba5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:06.880452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59725c43-85e0-4633-b9c0-8e6b28ebaba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:06.898525 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-59725c43-85e0-4633-b9c0-8e6b28ebaba5 None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:06.899990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-59725c43-85e0-4633-b9c0-8e6b28ebaba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:06.918634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1140/2286] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 06:03:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:06.932719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51209b3c-2ac0-4bc3-98e9-0f9c4cd9be10 service swift] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:06.944634 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51209b3c-2ac0-4bc3-98e9-0f9c4cd9be10 service swift] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:06.950203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4082d5-ebec-4b24-b106-6f7d4869e2b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:06.950893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4082d5-ebec-4b24-b106-6f7d4869e2b7 service swift] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:06.951179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4082d5-ebec-4b24-b106-6f7d4869e2b7 service swift] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:06.951456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4082d5-ebec-4b24-b106-6f7d4869e2b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:06.961375 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4082d5-ebec-4b24-b106-6f7d4869e2b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:06.963010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4082d5-ebec-4b24-b106-6f7d4869e2b7 service swift] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:06.972967 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1147/2287] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:03:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:03:07.571789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbc339-e61f-4df1-b6bc-77ce45b7221b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:07.572077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbc339-e61f-4df1-b6bc-77ce45b7221b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:07.577084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbc339-e61f-4df1-b6bc-77ce45b7221b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:07.589568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-c6cbc339-e61f-4df1-b6bc-77ce45b7221b None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:07.590849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c6cbc339-e61f-4df1-b6bc-77ce45b7221b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:07.598585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1141/2288] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 06:03:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:07.610605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4082d5-ebec-4b24-b106-6f7d4869e2b7 service swift] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:07.621596 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4082d5-ebec-4b24-b106-6f7d4869e2b7 service swift] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:07.626824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3d308-77d4-42fc-a3cb-64c591d7a60c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:07.627478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3d308-77d4-42fc-a3cb-64c591d7a60c service swift] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:07.627766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3d308-77d4-42fc-a3cb-64c591d7a60c service swift] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:07.628155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3d308-77d4-42fc-a3cb-64c591d7a60c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:07.637384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3d308-77d4-42fc-a3cb-64c591d7a60c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:07.638999 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3d308-77d4-42fc-a3cb-64c591d7a60c service swift] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:07.647283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1148/2289] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:03:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:03:08.348298 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd4615-835c-42f0-81fc-0a9538575450 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:08.348805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd4615-835c-42f0-81fc-0a9538575450 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:08.348898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd4615-835c-42f0-81fc-0a9538575450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:08.367372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9cbd4615-835c-42f0-81fc-0a9538575450 None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:08.368820 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9cbd4615-835c-42f0-81fc-0a9538575450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:08.380473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1142/2290] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 06:03:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:08.395225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3d308-77d4-42fc-a3cb-64c591d7a60c service swift] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:08.406337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3d308-77d4-42fc-a3cb-64c591d7a60c service swift] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:08.411812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48df618-f798-4155-854d-492fffd6c09c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:08.412530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48df618-f798-4155-854d-492fffd6c09c service swift] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:08.412878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48df618-f798-4155-854d-492fffd6c09c service swift] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:08.413221 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48df618-f798-4155-854d-492fffd6c09c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:08.429035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48df618-f798-4155-854d-492fffd6c09c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:08.430666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48df618-f798-4155-854d-492fffd6c09c service swift] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:08.439546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1149/2291] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:03:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:03:11.295423 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd4615-835c-42f0-81fc-0a9538575450 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:11.296839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9cbd4615-835c-42f0-81fc-0a9538575450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:11.360213 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd4615-835c-42f0-81fc-0a9538575450 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:11.364739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:11.366113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:11.366397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:11.366704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] PATH_INFO: `/v3/users/a3e574d24a714fc4a0a2877b63829f26` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:11.367914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e574d24a714fc4a0a2877b63829f26)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:11.369834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:11.548115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e574d24a714fc4a0a2877b63829f26'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:11.556716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e574d24a714fc4a0a2877b63829f26'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:11.629589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e574d24a714fc4a0a2877b63829f26'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:11.630254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e574d24a714fc4a0a2877b63829f26'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:11.630970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e574d24a714fc4a0a2877b63829f26'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:11.649392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1143/2292] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:03:11 2022] DELETE /identity/v3/users/a3e574d24a714fc4a0a2877b63829f26 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:03:11.655524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dda90a-c79f-4ec6-a6d4-5c4f9b48c90e None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:11.656046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dda90a-c79f-4ec6-a6d4-5c4f9b48c90e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:11.656160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dda90a-c79f-4ec6-a6d4-5c4f9b48c90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:11.675934 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-43dda90a-c79f-4ec6-a6d4-5c4f9b48c90e None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:11.677143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-43dda90a-c79f-4ec6-a6d4-5c4f9b48c90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:11.739165 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:11.753413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1150/2293] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:11.763552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:11.764967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:11.772287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:11.824475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6777d8-b412-422a-8def-ffe1f0d7aa60 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:11.829765 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b162c-b886-4af2-a5a1-c0d6ad9870a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:11.830669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b162c-b886-4af2-a5a1-c0d6ad9870a3 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:11.830834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b162c-b886-4af2-a5a1-c0d6ad9870a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:11.831115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b162c-b886-4af2-a5a1-c0d6ad9870a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:11.841782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b162c-b886-4af2-a5a1-c0d6ad9870a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:11.843283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b162c-b886-4af2-a5a1-c0d6ad9870a3 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:11.853263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1144/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:03:12.118394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dda90a-c79f-4ec6-a6d4-5c4f9b48c90e None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:12.119919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-43dda90a-c79f-4ec6-a6d4-5c4f9b48c90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:12.140963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dda90a-c79f-4ec6-a6d4-5c4f9b48c90e None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:12.147709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:12.148936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:12.149368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:12.149787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] PATH_INFO: `/v3/projects/30bfa024bb4a475e9f914d91f6d01a8a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:12.151505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30bfa024bb4a475e9f914d91f6d01a8a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:12.153640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:12.291059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30bfa024bb4a475e9f914d91f6d01a8a', 'request_id': 'req-08cf1cf7-85d0-41ee-8627-7e1137d0a077'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:12.297407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30bfa024bb4a475e9f914d91f6d01a8a', 'request_id': 'req-08cf1cf7-85d0-41ee-8627-7e1137d0a077'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:12.305315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30bfa024bb4a475e9f914d91f6d01a8a', 'request_id': 'req-08cf1cf7-85d0-41ee-8627-7e1137d0a077'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:12.405832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] The token cache is being invalidate because project 30bfa024bb4a475e9f914d91f6d01a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:03:12.406240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:12.406937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-08cf1cf7-85d0-41ee-8627-7e1137d0a077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:12.408489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1151/2295] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:03:12 2022] DELETE /identity/v3/projects/30bfa024bb4a475e9f914d91f6d01a8a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:03:22.191955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=24 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:22.402753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:27.479348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f99cb7-05a9-47fa-aaa2-9c6b6cabdbfe None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:27.479844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f99cb7-05a9-47fa-aaa2-9c6b6cabdbfe None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:27.479935 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f99cb7-05a9-47fa-aaa2-9c6b6cabdbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:27.497402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-98f99cb7-05a9-47fa-aaa2-9c6b6cabdbfe None None] MFA Rules not processed for user `7f269cd38f734209ba0496cf70469608`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:27.498644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-98f99cb7-05a9-47fa-aaa2-9c6b6cabdbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:27.554313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1145/2296] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:03:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:30.725675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04509f53-cae1-4a82-8225-d8fbb8665638 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:30.725675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04509f53-cae1-4a82-8225-d8fbb8665638 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:30.725675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04509f53-cae1-4a82-8225-d8fbb8665638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:30.751399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-04509f53-cae1-4a82-8225-d8fbb8665638 None None] MFA Rules not processed for user `13e5d648af294c7cbc86ccd362914728`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:30.752827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-04509f53-cae1-4a82-8225-d8fbb8665638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:30.816096 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1152/2297] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:03:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:03:34.888457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f99cb7-05a9-47fa-aaa2-9c6b6cabdbfe None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:34.889895 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-98f99cb7-05a9-47fa-aaa2-9c6b6cabdbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:34.950484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f99cb7-05a9-47fa-aaa2-9c6b6cabdbfe None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:34.954104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d053f2-1001-4842-bd8d-0cbac08baa71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:34.954785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d053f2-1001-4842-bd8d-0cbac08baa71 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:34.955073 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d053f2-1001-4842-bd8d-0cbac08baa71 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:34.955350 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d053f2-1001-4842-bd8d-0cbac08baa71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:34.957171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-13d053f2-1001-4842-bd8d-0cbac08baa71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:35.027790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d053f2-1001-4842-bd8d-0cbac08baa71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:35.029235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d053f2-1001-4842-bd8d-0cbac08baa71 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:35.041048 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1146/2298] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:03:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:03:38.416331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04509f53-cae1-4a82-8225-d8fbb8665638 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:38.418990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-04509f53-cae1-4a82-8225-d8fbb8665638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:38.489551 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04509f53-cae1-4a82-8225-d8fbb8665638 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:38.494904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:38.495785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:38.496066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:38.496340 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] PATH_INFO: `/v3/users/99feeb6cf0fa46b4aabecefcdf56deff` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:38.497562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99feeb6cf0fa46b4aabecefcdf56deff)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:38.499588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:38.725907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99feeb6cf0fa46b4aabecefcdf56deff'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:38.733353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99feeb6cf0fa46b4aabecefcdf56deff'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:38.734023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99feeb6cf0fa46b4aabecefcdf56deff'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:38.798041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99feeb6cf0fa46b4aabecefcdf56deff'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:38.798734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99feeb6cf0fa46b4aabecefcdf56deff'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:38.818680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1153/2299] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:03:38 2022] DELETE /identity/v3/users/99feeb6cf0fa46b4aabecefcdf56deff => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:03:39.074519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d053f2-1001-4842-bd8d-0cbac08baa71 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:39.084925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d053f2-1001-4842-bd8d-0cbac08baa71 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:39.132903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:39.147648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:39.148566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:39.148897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:39.149172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] PATH_INFO: `/v3/projects/d9f37a52ba6146e5b1cccfdba0c60a18` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:39.150559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f37a52ba6146e5b1cccfdba0c60a18)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:39.152660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:39.298299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f37a52ba6146e5b1cccfdba0c60a18', 'request_id': 'req-e086bd90-56ef-4238-a67f-d713993dbc71'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:39.305927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f37a52ba6146e5b1cccfdba0c60a18', 'request_id': 'req-e086bd90-56ef-4238-a67f-d713993dbc71'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:39.312607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f37a52ba6146e5b1cccfdba0c60a18', 'request_id': 'req-e086bd90-56ef-4238-a67f-d713993dbc71'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:39.391360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] The token cache is being invalidate because project d9f37a52ba6146e5b1cccfdba0c60a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:03:39.391655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:39.392420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e086bd90-56ef-4238-a67f-d713993dbc71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:39.394054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1147/2300] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:03:39 2022] DELETE /identity/v3/projects/d9f37a52ba6146e5b1cccfdba0c60a18 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:03:39.808236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:39.809812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:39.838737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:39.867485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107187c9-230c-485c-a436-b5c9568d79b6 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:39.916335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5e86c-e2f6-41c3-b616-1c40d29583dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:39.916989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5e86c-e2f6-41c3-b616-1c40d29583dc service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:39.917267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5e86c-e2f6-41c3-b616-1c40d29583dc service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:39.917537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5e86c-e2f6-41c3-b616-1c40d29583dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:39.919329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b6c5e86c-e2f6-41c3-b616-1c40d29583dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:39.988545 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5e86c-e2f6-41c3-b616-1c40d29583dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:39.989985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5e86c-e2f6-41c3-b616-1c40d29583dc service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:40.002130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1154/2301] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:03:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:03:49.128400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45f1af-23e9-4016-9cc6-885619a63f4b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:49.128926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45f1af-23e9-4016-9cc6-885619a63f4b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:49.129040 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45f1af-23e9-4016-9cc6-885619a63f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:49.147723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9e45f1af-23e9-4016-9cc6-885619a63f4b None None] MFA Rules not processed for user `dc8b3ecc32c14e31b55156586129a40c`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:49.148919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9e45f1af-23e9-4016-9cc6-885619a63f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:49.203756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1148/2302] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:03:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:49.996953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=63 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:50.966301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5e86c-e2f6-41c3-b616-1c40d29583dc service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:50.967665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b6c5e86c-e2f6-41c3-b616-1c40d29583dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:50.982323 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5e86c-e2f6-41c3-b616-1c40d29583dc service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:50.987849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb9ccb-3d86-4377-846c-30b2cd2de0ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:50.988492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb9ccb-3d86-4377-846c-30b2cd2de0ba service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:50.988990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb9ccb-3d86-4377-846c-30b2cd2de0ba service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:50.989136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb9ccb-3d86-4377-846c-30b2cd2de0ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:50.990781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-92fb9ccb-3d86-4377-846c-30b2cd2de0ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:51.001319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb9ccb-3d86-4377-846c-30b2cd2de0ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:51.002923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb9ccb-3d86-4377-846c-30b2cd2de0ba service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:51.011869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1155/2303] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:03:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:03:59.197253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:01.007708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:04.060490 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45f1af-23e9-4016-9cc6-885619a63f4b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:04.073735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45f1af-23e9-4016-9cc6-885619a63f4b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:04.079342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea61ae-7cfc-4cff-b489-b1d5cecd1830 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:04.079883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea61ae-7cfc-4cff-b489-b1d5cecd1830 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:04.080162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea61ae-7cfc-4cff-b489-b1d5cecd1830 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:04.080438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea61ae-7cfc-4cff-b489-b1d5cecd1830 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:04.082227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e5ea61ae-7cfc-4cff-b489-b1d5cecd1830 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:04.140152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea61ae-7cfc-4cff-b489-b1d5cecd1830 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:04.141747 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea61ae-7cfc-4cff-b489-b1d5cecd1830 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:04.150800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1149/2304] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:05.342387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb9ccb-3d86-4377-846c-30b2cd2de0ba service nova] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:05.344000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-92fb9ccb-3d86-4377-846c-30b2cd2de0ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:05.355456 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb9ccb-3d86-4377-846c-30b2cd2de0ba service nova] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:05.359978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c569c2-2166-482b-8826-24785185003d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:05.362907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c569c2-2166-482b-8826-24785185003d service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:05.362907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c569c2-2166-482b-8826-24785185003d service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:05.362907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c569c2-2166-482b-8826-24785185003d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:05.363214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b5c569c2-2166-482b-8826-24785185003d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:05.372856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c569c2-2166-482b-8826-24785185003d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:05.374887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c569c2-2166-482b-8826-24785185003d service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:05.383078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1156/2305] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:14.146567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=28 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:15.041407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea61ae-7cfc-4cff-b489-b1d5cecd1830 service nova] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:15.042835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e5ea61ae-7cfc-4cff-b489-b1d5cecd1830 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:15.098553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea61ae-7cfc-4cff-b489-b1d5cecd1830 service nova] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:15.102101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a553bd0-d3f8-43ab-9c67-2cdc1d8ab6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:15.102768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a553bd0-d3f8-43ab-9c67-2cdc1d8ab6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:15.103051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a553bd0-d3f8-43ab-9c67-2cdc1d8ab6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:15.103335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a553bd0-d3f8-43ab-9c67-2cdc1d8ab6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:15.105080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6a553bd0-d3f8-43ab-9c67-2cdc1d8ab6f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:15.169637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a553bd0-d3f8-43ab-9c67-2cdc1d8ab6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:15.171039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a553bd0-d3f8-43ab-9c67-2cdc1d8ab6f4 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:15.180648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1150/2306] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:15.379875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:23.755158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c569c2-2166-482b-8826-24785185003d service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:23.767795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c569c2-2166-482b-8826-24785185003d service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:23.780370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6f4ad-b3eb-4632-adaa-74f2dfa21659 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:23.781054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6f4ad-b3eb-4632-adaa-74f2dfa21659 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:23.783005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6f4ad-b3eb-4632-adaa-74f2dfa21659 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:23.783005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6f4ad-b3eb-4632-adaa-74f2dfa21659 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:23.787965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-60d6f4ad-b3eb-4632-adaa-74f2dfa21659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:23.815735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6f4ad-b3eb-4632-adaa-74f2dfa21659 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:23.822281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6f4ad-b3eb-4632-adaa-74f2dfa21659 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:23.850714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1157/2307] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:25.176424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=48 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:28.493442 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2bfbc-875f-4fa1-99a5-9f45dc8657fb None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:28.494000 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2bfbc-875f-4fa1-99a5-9f45dc8657fb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:28.494000 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2bfbc-875f-4fa1-99a5-9f45dc8657fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:28.511806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-55f2bfbc-875f-4fa1-99a5-9f45dc8657fb None None] MFA Rules not processed for user `6dc49525e3394668b4f1610c4badc144`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:28.513011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-55f2bfbc-875f-4fa1-99a5-9f45dc8657fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:28.520079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1151/2308] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:04:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:04:28.822680 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6f4ad-b3eb-4632-adaa-74f2dfa21659 service nova] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:28.824087 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-60d6f4ad-b3eb-4632-adaa-74f2dfa21659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:28.835123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6f4ad-b3eb-4632-adaa-74f2dfa21659 service nova] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:28.838808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68061c1-26b4-441a-a70d-54de36ed2086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:28.839446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68061c1-26b4-441a-a70d-54de36ed2086 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:28.839722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68061c1-26b4-441a-a70d-54de36ed2086 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:28.839993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68061c1-26b4-441a-a70d-54de36ed2086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:28.841681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b68061c1-26b4-441a-a70d-54de36ed2086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:28.851403 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68061c1-26b4-441a-a70d-54de36ed2086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:28.852837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68061c1-26b4-441a-a70d-54de36ed2086 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:28.861166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1158/2309] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:29.241402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2bfbc-875f-4fa1-99a5-9f45dc8657fb None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:29.243066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-55f2bfbc-875f-4fa1-99a5-9f45dc8657fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:29.314347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2bfbc-875f-4fa1-99a5-9f45dc8657fb None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:29.319268 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64febf-0447-4e32-9045-b3d39b39fd2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:29.319915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64febf-0447-4e32-9045-b3d39b39fd2f service placement] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:29.320229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64febf-0447-4e32-9045-b3d39b39fd2f service placement] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:29.320505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64febf-0447-4e32-9045-b3d39b39fd2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:29.322267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2c64febf-0447-4e32-9045-b3d39b39fd2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:29.333793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64febf-0447-4e32-9045-b3d39b39fd2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:29.335428 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64febf-0447-4e32-9045-b3d39b39fd2f service placement] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:29.345699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1152/2310] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:29.870276 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68061c1-26b4-441a-a70d-54de36ed2086 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:29.885975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68061c1-26b4-441a-a70d-54de36ed2086 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:29.894729 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184429d2-8797-498c-b830-fa862b4296f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:29.895398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184429d2-8797-498c-b830-fa862b4296f4 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:29.895749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184429d2-8797-498c-b830-fa862b4296f4 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:29.896025 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184429d2-8797-498c-b830-fa862b4296f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:29.897853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-184429d2-8797-498c-b830-fa862b4296f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:29.959055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184429d2-8797-498c-b830-fa862b4296f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:29.961515 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184429d2-8797-498c-b830-fa862b4296f4 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:29.975103 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1159/2311] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:38.835339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64febf-0447-4e32-9045-b3d39b39fd2f service placement] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:38.844780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64febf-0447-4e32-9045-b3d39b39fd2f service placement] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:38.848407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21968ef8-0b44-4bcb-934d-e041b8ff750b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:38.849044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21968ef8-0b44-4bcb-934d-e041b8ff750b service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:38.849372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21968ef8-0b44-4bcb-934d-e041b8ff750b service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:38.849650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21968ef8-0b44-4bcb-934d-e041b8ff750b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:38.851439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-21968ef8-0b44-4bcb-934d-e041b8ff750b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:38.920795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21968ef8-0b44-4bcb-934d-e041b8ff750b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:38.922314 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21968ef8-0b44-4bcb-934d-e041b8ff750b service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:38.932785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1153/2312] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:39.966957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=40 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:42.578069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184429d2-8797-498c-b830-fa862b4296f4 service nova] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:42.580420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-184429d2-8797-498c-b830-fa862b4296f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:42.599647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184429d2-8797-498c-b830-fa862b4296f4 service nova] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:42.610557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809fb29-db2c-4b19-806f-805a1de261c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:42.611297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809fb29-db2c-4b19-806f-805a1de261c8 service placement] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:42.611589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809fb29-db2c-4b19-806f-805a1de261c8 service placement] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:42.612034 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809fb29-db2c-4b19-806f-805a1de261c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:42.614430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a809fb29-db2c-4b19-806f-805a1de261c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:42.629499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809fb29-db2c-4b19-806f-805a1de261c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:42.633952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809fb29-db2c-4b19-806f-805a1de261c8 service placement] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:42.639844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21968ef8-0b44-4bcb-934d-e041b8ff750b service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:42.654688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1160/2313] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:42.670062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21968ef8-0b44-4bcb-934d-e041b8ff750b service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:42.677486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7588ce-df39-4a12-8d84-4e4451bb6d90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:42.678330 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7588ce-df39-4a12-8d84-4e4451bb6d90 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:42.678713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7588ce-df39-4a12-8d84-4e4451bb6d90 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:42.679123 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7588ce-df39-4a12-8d84-4e4451bb6d90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:42.681638 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4f7588ce-df39-4a12-8d84-4e4451bb6d90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:42.696791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7588ce-df39-4a12-8d84-4e4451bb6d90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:42.698446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7588ce-df39-4a12-8d84-4e4451bb6d90 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:42.710459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1154/2314] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:42.981776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809fb29-db2c-4b19-806f-805a1de261c8 service placement] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:42.983595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a809fb29-db2c-4b19-806f-805a1de261c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:42.999565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809fb29-db2c-4b19-806f-805a1de261c8 service placement] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:43.004846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:43.005732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:43.006019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:43.006298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] PATH_INFO: `/v3/users/13e5d648af294c7cbc86ccd362914728` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:43.007539 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e5d648af294c7cbc86ccd362914728)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:43.009857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:43.196593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e5d648af294c7cbc86ccd362914728'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:43.203939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e5d648af294c7cbc86ccd362914728'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:43.204774 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e5d648af294c7cbc86ccd362914728'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:43.262891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e5d648af294c7cbc86ccd362914728'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:43.264133 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d1e201af-d541-4b8e-807d-c5af7d3cc053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e5d648af294c7cbc86ccd362914728'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:43.283210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1161/2315] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:04:42 2022] DELETE /identity/v3/users/13e5d648af294c7cbc86ccd362914728 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:04:43.523433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7588ce-df39-4a12-8d84-4e4451bb6d90 service nova] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:43.536147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7588ce-df39-4a12-8d84-4e4451bb6d90 service nova] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:43.602086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:43.602995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:43.603301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:43.603595 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] PATH_INFO: `/v3/projects/d30518907b9f4939b1e8666eba12186e` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:43.604779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30518907b9f4939b1e8666eba12186e)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:43.606753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:43.613154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:43.731272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30518907b9f4939b1e8666eba12186e', 'request_id': 'req-03cbff21-272a-46ce-8186-03ba7723cdd7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:43.737238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30518907b9f4939b1e8666eba12186e', 'request_id': 'req-03cbff21-272a-46ce-8186-03ba7723cdd7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:43.742593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30518907b9f4939b1e8666eba12186e', 'request_id': 'req-03cbff21-272a-46ce-8186-03ba7723cdd7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:43.831959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] The token cache is being invalidate because project d30518907b9f4939b1e8666eba12186e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:04:43.832178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:43.832824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cbff21-272a-46ce-8186-03ba7723cdd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:43.834363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1155/2316] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:04:43 2022] DELETE /identity/v3/projects/d30518907b9f4939b1e8666eba12186e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:04:43.839354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742ccab-5a12-4f4f-bc83-5b6ce521d087 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:43.839861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742ccab-5a12-4f4f-bc83-5b6ce521d087 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:43.839967 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742ccab-5a12-4f4f-bc83-5b6ce521d087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:43.858310 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8742ccab-5a12-4f4f-bc83-5b6ce521d087 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:43.860070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8742ccab-5a12-4f4f-bc83-5b6ce521d087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:43.926985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1162/2317] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:04:43.935074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55daf-af42-4520-8eb0-c053c0be6cea None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:43.935364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55daf-af42-4520-8eb0-c053c0be6cea None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:43.935646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55daf-af42-4520-8eb0-c053c0be6cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:43.954462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-ccf55daf-af42-4520-8eb0-c053c0be6cea None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:43.956371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ccf55daf-af42-4520-8eb0-c053c0be6cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:43.964787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1156/2318] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:04:43.968750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742ccab-5a12-4f4f-bc83-5b6ce521d087 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:43.978973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742ccab-5a12-4f4f-bc83-5b6ce521d087 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:43.983956 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1f4fb-db58-45c1-b768-e647086e3b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:43.984661 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1f4fb-db58-45c1-b768-e647086e3b84 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:43.984983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1f4fb-db58-45c1-b768-e647086e3b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:43.985269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1f4fb-db58-45c1-b768-e647086e3b84 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:43.985996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1f4fb-db58-45c1-b768-e647086e3b84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:04:43.986304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1f4fb-db58-45c1-b768-e647086e3b84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:43.987937 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1f4fb-db58-45c1-b768-e647086e3b84 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:44.000488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1163/2319] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:04:44.005023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf55daf-af42-4520-8eb0-c053c0be6cea None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:44.015920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf55daf-af42-4520-8eb0-c053c0be6cea None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:44.021311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b9bf7-7de1-4f6b-855e-d4e2d2ee7e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:44.022045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b9bf7-7de1-4f6b-855e-d4e2d2ee7e4a admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:44.022333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b9bf7-7de1-4f6b-855e-d4e2d2ee7e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:44.022645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b9bf7-7de1-4f6b-855e-d4e2d2ee7e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:44.023343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b9bf7-7de1-4f6b-855e-d4e2d2ee7e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:44.025508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b9bf7-7de1-4f6b-855e-d4e2d2ee7e4a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:44.140263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1157/2320] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:04:44 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:04:44.144892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1f4fb-db58-45c1-b768-e647086e3b84 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:44.159451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1f4fb-db58-45c1-b768-e647086e3b84 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:44.209006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:44.221752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be05a9f-5870-4fdd-8eb9-517e88148ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:44.222586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be05a9f-5870-4fdd-8eb9-517e88148ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:44.222867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be05a9f-5870-4fdd-8eb9-517e88148ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:44.223574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be05a9f-5870-4fdd-8eb9-517e88148ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:44.223841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be05a9f-5870-4fdd-8eb9-517e88148ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:44.225755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be05a9f-5870-4fdd-8eb9-517e88148ad5 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:44.291564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1164/2321] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:04:44 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:04:44.295980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b9bf7-7de1-4f6b-855e-d4e2d2ee7e4a admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:44.306837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b9bf7-7de1-4f6b-855e-d4e2d2ee7e4a admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:44.311408 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fd447-bcb8-4a89-a654-87511efcf309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:44.312057 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fd447-bcb8-4a89-a654-87511efcf309 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:44.312337 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fd447-bcb8-4a89-a654-87511efcf309 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:44.312609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fd447-bcb8-4a89-a654-87511efcf309 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:44.313447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fd447-bcb8-4a89-a654-87511efcf309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:44.315737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fd447-bcb8-4a89-a654-87511efcf309 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:44.325977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1158/2322] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:04:44.329991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be05a9f-5870-4fdd-8eb9-517e88148ad5 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:44.340698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be05a9f-5870-4fdd-8eb9-517e88148ad5 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:44.344898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179fd43-f9b0-4bf1-a3c0-6184c112a3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:44.345564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179fd43-f9b0-4bf1-a3c0-6184c112a3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:44.345876 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179fd43-f9b0-4bf1-a3c0-6184c112a3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:44.346152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179fd43-f9b0-4bf1-a3c0-6184c112a3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:44.346907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179fd43-f9b0-4bf1-a3c0-6184c112a3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:44.348583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179fd43-f9b0-4bf1-a3c0-6184c112a3c0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:44.361302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1165/2323] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:04:44.366131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fd447-bcb8-4a89-a654-87511efcf309 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:44.376435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fd447-bcb8-4a89-a654-87511efcf309 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:44.381766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4fc7c-7a9c-458d-bae3-59dd9faf2809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:44.382665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4fc7c-7a9c-458d-bae3-59dd9faf2809 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:44.382943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4fc7c-7a9c-458d-bae3-59dd9faf2809 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:44.383215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4fc7c-7a9c-458d-bae3-59dd9faf2809 admin admin] PATH_INFO: `/v3/projects/bd96ee296dc9494489f275f9ae3c5a39/users/567799f2371a47d6841a2dc4621939d7/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:44.397914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4fc7c-7a9c-458d-bae3-59dd9faf2809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd96ee296dc9494489f275f9ae3c5a39, user_id=567799f2371a47d6841a2dc4621939d7, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:44.400655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4fc7c-7a9c-458d-bae3-59dd9faf2809 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:44.476966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1159/2324] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:04:44 2022] PUT /identity/v3/projects/bd96ee296dc9494489f275f9ae3c5a39/users/567799f2371a47d6841a2dc4621939d7/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:04:44.487719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9969176-1483-424d-a8cd-b6882edf452a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:44.488009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9969176-1483-424d-a8cd-b6882edf452a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:44.488288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9969176-1483-424d-a8cd-b6882edf452a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:44.517657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-b9969176-1483-424d-a8cd-b6882edf452a None None] MFA Rules not processed for user `567799f2371a47d6841a2dc4621939d7`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:44.519082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b9969176-1483-424d-a8cd-b6882edf452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:44.584383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1166/2325] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:04:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:04:44.588826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4fc7c-7a9c-458d-bae3-59dd9faf2809 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:44.650384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4fc7c-7a9c-458d-bae3-59dd9faf2809 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:44.654665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f30af1-baa6-44c4-be4b-1576e96e2d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:44.655407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f30af1-baa6-44c4-be4b-1576e96e2d13 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:44.655761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f30af1-baa6-44c4-be4b-1576e96e2d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:44.656043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f30af1-baa6-44c4-be4b-1576e96e2d13 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:44.656751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f30af1-baa6-44c4-be4b-1576e96e2d13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:44.659180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f30af1-baa6-44c4-be4b-1576e96e2d13 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:44.771541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1160/2326] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:04:44 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:04:44.776617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9969176-1483-424d-a8cd-b6882edf452a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:44.793250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9969176-1483-424d-a8cd-b6882edf452a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:44.853364 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ebf99-7b60-4041-855a-be2de035b3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:44.854170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ebf99-7b60-4041-855a-be2de035b3ad admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:44.854619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ebf99-7b60-4041-855a-be2de035b3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:44.854905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ebf99-7b60-4041-855a-be2de035b3ad admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:44.855717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ebf99-7b60-4041-855a-be2de035b3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:44.857832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ebf99-7b60-4041-855a-be2de035b3ad admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:44.989551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f30af1-baa6-44c4-be4b-1576e96e2d13 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:44.993403 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1167/2327] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:04:44 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:04:45.001178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f30af1-baa6-44c4-be4b-1576e96e2d13 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:45.006909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e12453-dbe9-46cd-8ea7-a2ff27aeb6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:45.007776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e12453-dbe9-46cd-8ea7-a2ff27aeb6dc admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:45.008243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e12453-dbe9-46cd-8ea7-a2ff27aeb6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:45.008685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e12453-dbe9-46cd-8ea7-a2ff27aeb6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:45.009856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e12453-dbe9-46cd-8ea7-a2ff27aeb6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:45.012336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e12453-dbe9-46cd-8ea7-a2ff27aeb6dc admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:45.023455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1161/2328] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:04:45.027783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ebf99-7b60-4041-855a-be2de035b3ad admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:45.037714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ebf99-7b60-4041-855a-be2de035b3ad admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:45.042641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7463c-843c-483d-ac0c-62a2155be507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:45.043529 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7463c-843c-483d-ac0c-62a2155be507 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:45.043801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7463c-843c-483d-ac0c-62a2155be507 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:45.044074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7463c-843c-483d-ac0c-62a2155be507 admin admin] PATH_INFO: `/v3/projects/fc6445d2506142ae95003a5dea8a3052/users/048b1b8bdc2f4f81954bbd5f802309d8/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:45.060807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7463c-843c-483d-ac0c-62a2155be507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc6445d2506142ae95003a5dea8a3052, user_id=048b1b8bdc2f4f81954bbd5f802309d8, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:45.063629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7463c-843c-483d-ac0c-62a2155be507 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:45.177908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1168/2329] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:04:45 2022] PUT /identity/v3/projects/fc6445d2506142ae95003a5dea8a3052/users/048b1b8bdc2f4f81954bbd5f802309d8/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:04:45.181934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e12453-dbe9-46cd-8ea7-a2ff27aeb6dc admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:45.188282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:45.192817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e12453-dbe9-46cd-8ea7-a2ff27aeb6dc admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:45.247556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bc841-12a7-42c8-a9b8-0ddf9c8f5873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:45.248191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bc841-12a7-42c8-a9b8-0ddf9c8f5873 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:45.248470 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bc841-12a7-42c8-a9b8-0ddf9c8f5873 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:45.248745 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bc841-12a7-42c8-a9b8-0ddf9c8f5873 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:45.249474 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bc841-12a7-42c8-a9b8-0ddf9c8f5873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:45.251176 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bc841-12a7-42c8-a9b8-0ddf9c8f5873 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:45.265899 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1162/2330] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:04:45.267199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7463c-843c-483d-ac0c-62a2155be507 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:45.277847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7463c-843c-483d-ac0c-62a2155be507 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:45.282444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13321f89-f987-45c6-afcb-363a515ba538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:45.283356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13321f89-f987-45c6-afcb-363a515ba538 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:45.284267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13321f89-f987-45c6-afcb-363a515ba538 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:45.284542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13321f89-f987-45c6-afcb-363a515ba538 admin admin] PATH_INFO: `/v3/projects/fc6445d2506142ae95003a5dea8a3052/users/048b1b8bdc2f4f81954bbd5f802309d8/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:45.286605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13321f89-f987-45c6-afcb-363a515ba538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc6445d2506142ae95003a5dea8a3052, user_id=048b1b8bdc2f4f81954bbd5f802309d8, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:45.289198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13321f89-f987-45c6-afcb-363a515ba538 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:45.300924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1169/2331] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:04:45 2022] PUT /identity/v3/projects/fc6445d2506142ae95003a5dea8a3052/users/048b1b8bdc2f4f81954bbd5f802309d8/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:04:45.310967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ddbb-37d9-4995-9baf-715eacb02dfb None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:45.311957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ddbb-37d9-4995-9baf-715eacb02dfb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:45.311957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ddbb-37d9-4995-9baf-715eacb02dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:45.338764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-fc59ddbb-37d9-4995-9baf-715eacb02dfb None None] MFA Rules not processed for user `048b1b8bdc2f4f81954bbd5f802309d8`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:45.342716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-fc59ddbb-37d9-4995-9baf-715eacb02dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:45.417738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1163/2332] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:04:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:04:45.432668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13321f89-f987-45c6-afcb-363a515ba538 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:45.432668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-13321f89-f987-45c6-afcb-363a515ba538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:45.440915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:45.487219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13321f89-f987-45c6-afcb-363a515ba538 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:45.490889 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2f0c9-9add-4c01-875b-67db794f7237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:45.491535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2f0c9-9add-4c01-875b-67db794f7237 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:45.491815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2f0c9-9add-4c01-875b-67db794f7237 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:45.492090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2f0c9-9add-4c01-875b-67db794f7237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:45.550600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2f0c9-9add-4c01-875b-67db794f7237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:45.552056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2f0c9-9add-4c01-875b-67db794f7237 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:45.561063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1170/2333] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:46.898777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2565df1-a108-4206-bdf3-97bba08dd4e8 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:46.899271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2565df1-a108-4206-bdf3-97bba08dd4e8 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:46.899368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2565df1-a108-4206-bdf3-97bba08dd4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:46.919223 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f2565df1-a108-4206-bdf3-97bba08dd4e8 None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:46.920572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f2565df1-a108-4206-bdf3-97bba08dd4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:46.977857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1164/2334] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 06:04:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:04:47.162697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2f0c9-9add-4c01-875b-67db794f7237 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:47.164670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4ab2f0c9-9add-4c01-875b-67db794f7237 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:47.245805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2f0c9-9add-4c01-875b-67db794f7237 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:47.253432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b371f56-6d4a-4468-ac2c-4300af44d44a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:47.255046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371f56-6d4a-4468-ac2c-4300af44d44a service swift] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:47.255430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371f56-6d4a-4468-ac2c-4300af44d44a service swift] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:47.255812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371f56-6d4a-4468-ac2c-4300af44d44a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:47.268400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b371f56-6d4a-4468-ac2c-4300af44d44a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:47.270122 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b371f56-6d4a-4468-ac2c-4300af44d44a service swift] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:47.282072 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1171/2335] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:04:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:04:52.118754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2565df1-a108-4206-bdf3-97bba08dd4e8 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:52.132382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2565df1-a108-4206-bdf3-97bba08dd4e8 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:52.136497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c6956-c3f8-460e-9707-4869eb45cf6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:52.137394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c6956-c3f8-460e-9707-4869eb45cf6f service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:52.137822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c6956-c3f8-460e-9707-4869eb45cf6f service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:52.138105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c6956-c3f8-460e-9707-4869eb45cf6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:52.149025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c6956-c3f8-460e-9707-4869eb45cf6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:52.150512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c6956-c3f8-460e-9707-4869eb45cf6f service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:52.161834 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1165/2336] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:57.278131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=77 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:02.160657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:14.653953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf43aa4-643e-474b-8292-c4aad36ecc4a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:14.654875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf43aa4-643e-474b-8292-c4aad36ecc4a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:14.655160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf43aa4-643e-474b-8292-c4aad36ecc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:14.677432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8cf43aa4-643e-474b-8292-c4aad36ecc4a None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:14.682625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8cf43aa4-643e-474b-8292-c4aad36ecc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:14.690978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1172/2337] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 06:05:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:14.712975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c6956-c3f8-460e-9707-4869eb45cf6f service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:14.725387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c6956-c3f8-460e-9707-4869eb45cf6f service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:14.731038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407fabc-9e53-4148-9a93-ef6ccc1ad8a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:14.732128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407fabc-9e53-4148-9a93-ef6ccc1ad8a3 service swift] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:14.732592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407fabc-9e53-4148-9a93-ef6ccc1ad8a3 service swift] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:14.733018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407fabc-9e53-4148-9a93-ef6ccc1ad8a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:14.744708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407fabc-9e53-4148-9a93-ef6ccc1ad8a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:14.746811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407fabc-9e53-4148-9a93-ef6ccc1ad8a3 service swift] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:14.758687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1166/2338] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:05:19.806037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf43aa4-643e-474b-8292-c4aad36ecc4a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:19.899106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf43aa4-643e-474b-8292-c4aad36ecc4a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:19.909060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:19.910135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:19.910423 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:19.910730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] PATH_INFO: `/v3/users/567799f2371a47d6841a2dc4621939d7` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:19.914058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=567799f2371a47d6841a2dc4621939d7)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:19.914058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:20.157136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '567799f2371a47d6841a2dc4621939d7'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:20.164762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '567799f2371a47d6841a2dc4621939d7'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:20.165480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '567799f2371a47d6841a2dc4621939d7'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:20.250838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '567799f2371a47d6841a2dc4621939d7'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:20.251647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '567799f2371a47d6841a2dc4621939d7'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:20.272319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1173/2339] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:19 2022] DELETE /identity/v3/users/567799f2371a47d6841a2dc4621939d7 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:20.277099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4e0e4-f388-4089-a076-9d0fda49a673 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:20.277975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4e0e4-f388-4089-a076-9d0fda49a673 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:20.278261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4e0e4-f388-4089-a076-9d0fda49a673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:20.297344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-92c4e0e4-f388-4089-a076-9d0fda49a673 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:20.298645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-92c4e0e4-f388-4089-a076-9d0fda49a673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:20.376700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1167/2340] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:20.384408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:20.385829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:20.453855 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe03134-be0c-42d0-b4a2-7f654c930b79 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:20.457511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38ff68-77cb-4c02-aee6-4cc4d9630174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:20.458172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38ff68-77cb-4c02-aee6-4cc4d9630174 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:20.458454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38ff68-77cb-4c02-aee6-4cc4d9630174 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:20.458823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38ff68-77cb-4c02-aee6-4cc4d9630174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:20.468603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38ff68-77cb-4c02-aee6-4cc4d9630174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:20.470065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38ff68-77cb-4c02-aee6-4cc4d9630174 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:20.491570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1174/2341] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:21.098019 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4e0e4-f388-4089-a076-9d0fda49a673 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:21.099760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-92c4e0e4-f388-4089-a076-9d0fda49a673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:21.116180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4e0e4-f388-4089-a076-9d0fda49a673 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:21.122405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:21.123543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:21.124213 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:21.124653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] PATH_INFO: `/v3/projects/bd96ee296dc9494489f275f9ae3c5a39` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:21.126057 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd96ee296dc9494489f275f9ae3c5a39)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:21.146774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:21.276195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd96ee296dc9494489f275f9ae3c5a39', 'request_id': 'req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:21.286824 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd96ee296dc9494489f275f9ae3c5a39', 'request_id': 'req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:21.292949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd96ee296dc9494489f275f9ae3c5a39', 'request_id': 'req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:21.347786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] The token cache is being invalidate because project bd96ee296dc9494489f275f9ae3c5a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:21.348384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:21.349514 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:21.351560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1168/2342] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:21 2022] DELETE /identity/v3/projects/bd96ee296dc9494489f275f9ae3c5a39 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:21.355683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38ff68-77cb-4c02-aee6-4cc4d9630174 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:21.366612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38ff68-77cb-4c02-aee6-4cc4d9630174 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:21.383621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:21.436182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:21.437182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:21.437462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:21.437765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] PATH_INFO: `/v3/users/048b1b8bdc2f4f81954bbd5f802309d8` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:21.439022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] RBAC: Authorizing `identity:delete_user(user_id=048b1b8bdc2f4f81954bbd5f802309d8)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:21.441499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:22.090395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '048b1b8bdc2f4f81954bbd5f802309d8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:22.103971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '048b1b8bdc2f4f81954bbd5f802309d8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:22.105169 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '048b1b8bdc2f4f81954bbd5f802309d8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:22.165421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '048b1b8bdc2f4f81954bbd5f802309d8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:22.165421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-43826699-521f-4228-8dc4-21790754c21c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '048b1b8bdc2f4f81954bbd5f802309d8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:22.202675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1175/2343] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:21 2022] DELETE /identity/v3/users/048b1b8bdc2f4f81954bbd5f802309d8 => generated 0 bytes in 847 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:22.676629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:22.682457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:22.694054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2c0c3-e991-4f0a-b6c5-6051ab505ec2 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:22.755110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:22.755991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:22.756273 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:22.756557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] PATH_INFO: `/v3/projects/fc6445d2506142ae95003a5dea8a3052` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:22.757755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc6445d2506142ae95003a5dea8a3052)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:22.759793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:22.772173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:22.878008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc6445d2506142ae95003a5dea8a3052', 'request_id': 'req-a38d785f-fc4d-45a5-ac1f-776977687587'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:22.883915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc6445d2506142ae95003a5dea8a3052', 'request_id': 'req-a38d785f-fc4d-45a5-ac1f-776977687587'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:22.889104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc6445d2506142ae95003a5dea8a3052', 'request_id': 'req-a38d785f-fc4d-45a5-ac1f-776977687587'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:22.974901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] The token cache is being invalidate because project fc6445d2506142ae95003a5dea8a3052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:22.975277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:22.975979 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-a38d785f-fc4d-45a5-ac1f-776977687587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:22.977574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1169/2344] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:22 2022] DELETE /identity/v3/projects/fc6445d2506142ae95003a5dea8a3052 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:22.982351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5473b-a101-4d21-a2dd-ce6bf33e98ff None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:22.982870 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5473b-a101-4d21-a2dd-ce6bf33e98ff None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:22.983049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5473b-a101-4d21-a2dd-ce6bf33e98ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:23.001669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-66c5473b-a101-4d21-a2dd-ce6bf33e98ff None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:23.004918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-66c5473b-a101-4d21-a2dd-ce6bf33e98ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:23.068014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1176/2345] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:23.075826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a8202-6395-4108-a241-9bd8685e2c0c None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:23.076341 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a8202-6395-4108-a241-9bd8685e2c0c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:23.076649 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a8202-6395-4108-a241-9bd8685e2c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:23.094854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-372a8202-6395-4108-a241-9bd8685e2c0c None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:23.096064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-372a8202-6395-4108-a241-9bd8685e2c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:23.104535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1170/2346] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:23.108727 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5473b-a101-4d21-a2dd-ce6bf33e98ff None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:23.119100 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5473b-a101-4d21-a2dd-ce6bf33e98ff None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:23.123722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3698c-f806-45a2-809b-b4c45d0159c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:23.124385 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3698c-f806-45a2-809b-b4c45d0159c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:23.124669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3698c-f806-45a2-809b-b4c45d0159c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:23.124949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3698c-f806-45a2-809b-b4c45d0159c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:23.125703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3698c-f806-45a2-809b-b4c45d0159c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:23.126014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3698c-f806-45a2-809b-b4c45d0159c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:23.127699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3698c-f806-45a2-809b-b4c45d0159c0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:23.145915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1177/2347] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:05:23.149055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a8202-6395-4108-a241-9bd8685e2c0c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:23.158184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a8202-6395-4108-a241-9bd8685e2c0c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:23.163392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c186844-24e0-45fe-be72-4c55995671f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:23.164045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c186844-24e0-45fe-be72-4c55995671f0 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:23.164325 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c186844-24e0-45fe-be72-4c55995671f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:23.164598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c186844-24e0-45fe-be72-4c55995671f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:23.165283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c186844-24e0-45fe-be72-4c55995671f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:23.167210 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c186844-24e0-45fe-be72-4c55995671f0 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:23.312473 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1171/2348] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:23 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:23.313661 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3698c-f806-45a2-809b-b4c45d0159c0 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:23.324980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3698c-f806-45a2-809b-b4c45d0159c0 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:23.346713 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:23.383808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d579f9-208d-4205-b429-54ffd83ec68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:23.384546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d579f9-208d-4205-b429-54ffd83ec68e admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:23.384834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d579f9-208d-4205-b429-54ffd83ec68e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:23.385114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d579f9-208d-4205-b429-54ffd83ec68e admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:23.385803 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d579f9-208d-4205-b429-54ffd83ec68e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:23.387746 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d579f9-208d-4205-b429-54ffd83ec68e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:23.495226 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1178/2349] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:23 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:23.499422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c186844-24e0-45fe-be72-4c55995671f0 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:23.510499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c186844-24e0-45fe-be72-4c55995671f0 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:23.515666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e9366-d677-48e6-984e-9e0cad9947de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:23.516338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e9366-d677-48e6-984e-9e0cad9947de admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:23.516640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e9366-d677-48e6-984e-9e0cad9947de admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:23.516918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e9366-d677-48e6-984e-9e0cad9947de admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:23.517704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e9366-d677-48e6-984e-9e0cad9947de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:23.519976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e9366-d677-48e6-984e-9e0cad9947de admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:23.531327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1172/2350] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:23.536014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d579f9-208d-4205-b429-54ffd83ec68e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:23.545931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d579f9-208d-4205-b429-54ffd83ec68e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:23.551397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd554a-8202-459b-b6a9-0a2d0483988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:23.552071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd554a-8202-459b-b6a9-0a2d0483988d admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:23.552353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd554a-8202-459b-b6a9-0a2d0483988d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:23.552632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd554a-8202-459b-b6a9-0a2d0483988d admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:23.553355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd554a-8202-459b-b6a9-0a2d0483988d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:23.555036 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd554a-8202-459b-b6a9-0a2d0483988d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:23.565324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1179/2351] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:05:23.569893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e9366-d677-48e6-984e-9e0cad9947de admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:23.579936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e9366-d677-48e6-984e-9e0cad9947de admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:23.585314 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2d4a2-7d19-44bc-ae33-62d6b124e0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:23.586197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2d4a2-7d19-44bc-ae33-62d6b124e0fc admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:23.586498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2d4a2-7d19-44bc-ae33-62d6b124e0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:23.586831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2d4a2-7d19-44bc-ae33-62d6b124e0fc admin admin] PATH_INFO: `/v3/projects/384c77590a9d4e64977bacaccce929ae/users/49b5ff6d18a945a78233ec48065bb60d/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:23.602869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2d4a2-7d19-44bc-ae33-62d6b124e0fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=384c77590a9d4e64977bacaccce929ae, user_id=49b5ff6d18a945a78233ec48065bb60d, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:23.605574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2d4a2-7d19-44bc-ae33-62d6b124e0fc admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:23.673025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1173/2352] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:23 2022] PUT /identity/v3/projects/384c77590a9d4e64977bacaccce929ae/users/49b5ff6d18a945a78233ec48065bb60d/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:23.683509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fe1ca-1af1-4670-aef0-fa8f39fa27ee None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:23.683800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fe1ca-1af1-4670-aef0-fa8f39fa27ee None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:23.684080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fe1ca-1af1-4670-aef0-fa8f39fa27ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:23.711506 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-501fe1ca-1af1-4670-aef0-fa8f39fa27ee None None] MFA Rules not processed for user `49b5ff6d18a945a78233ec48065bb60d`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:23.712671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-501fe1ca-1af1-4670-aef0-fa8f39fa27ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:23.775804 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1180/2353] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:23.780296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2d4a2-7d19-44bc-ae33-62d6b124e0fc admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:23.841013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2d4a2-7d19-44bc-ae33-62d6b124e0fc admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:23.845714 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d169c-f403-40f1-8afa-259844b2977e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:23.846362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d169c-f403-40f1-8afa-259844b2977e admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:23.846733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d169c-f403-40f1-8afa-259844b2977e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:23.847018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d169c-f403-40f1-8afa-259844b2977e admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:23.847705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d169c-f403-40f1-8afa-259844b2977e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:23.849658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d169c-f403-40f1-8afa-259844b2977e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:23.932671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1174/2354] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:23 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:23.936518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fe1ca-1af1-4670-aef0-fa8f39fa27ee None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:23.947799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fe1ca-1af1-4670-aef0-fa8f39fa27ee None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:24.007216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1e8b2-fd81-4c2b-888e-708e801b33ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:24.008128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1e8b2-fd81-4c2b-888e-708e801b33ac admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:24.008522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1e8b2-fd81-4c2b-888e-708e801b33ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:24.008931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1e8b2-fd81-4c2b-888e-708e801b33ac admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:24.009981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1e8b2-fd81-4c2b-888e-708e801b33ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:24.012984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1e8b2-fd81-4c2b-888e-708e801b33ac admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:24.121342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1181/2355] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:23 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:24.122591 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d169c-f403-40f1-8afa-259844b2977e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:24.133432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d169c-f403-40f1-8afa-259844b2977e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:24.138008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6c20f-084e-40d5-9fa7-7696687cbf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:24.138705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c20f-084e-40d5-9fa7-7696687cbf61 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:24.138987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c20f-084e-40d5-9fa7-7696687cbf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:24.139262 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c20f-084e-40d5-9fa7-7696687cbf61 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:24.139988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6c20f-084e-40d5-9fa7-7696687cbf61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:24.141671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6c20f-084e-40d5-9fa7-7696687cbf61 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:24.158074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1175/2356] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:24.162143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1e8b2-fd81-4c2b-888e-708e801b33ac admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:24.172561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1e8b2-fd81-4c2b-888e-708e801b33ac admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:24.177638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03a15e-41cc-40a2-8f05-473f8222679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:24.178300 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03a15e-41cc-40a2-8f05-473f8222679a admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:24.178616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03a15e-41cc-40a2-8f05-473f8222679a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:24.178894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03a15e-41cc-40a2-8f05-473f8222679a admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:24.179618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03a15e-41cc-40a2-8f05-473f8222679a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:24.181194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03a15e-41cc-40a2-8f05-473f8222679a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:24.191879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1182/2357] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:05:24.195965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6c20f-084e-40d5-9fa7-7696687cbf61 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:24.206222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6c20f-084e-40d5-9fa7-7696687cbf61 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:24.211179 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a480a4f-dae1-4371-abc9-402e84313cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:24.212064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a480a4f-dae1-4371-abc9-402e84313cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:24.212376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a480a4f-dae1-4371-abc9-402e84313cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:24.212651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a480a4f-dae1-4371-abc9-402e84313cb9 admin admin] PATH_INFO: `/v3/projects/d83e38aa761d414f8091b241dd6776fd/users/09c0fab0073545d48e6bf7755b5d4f18/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:24.220879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:24.227724 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a480a4f-dae1-4371-abc9-402e84313cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83e38aa761d414f8091b241dd6776fd, user_id=09c0fab0073545d48e6bf7755b5d4f18, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:24.230438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a480a4f-dae1-4371-abc9-402e84313cb9 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:24.329254 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1176/2358] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:24 2022] PUT /identity/v3/projects/d83e38aa761d414f8091b241dd6776fd/users/09c0fab0073545d48e6bf7755b5d4f18/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:24.341305 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc10a31-9d71-4a01-8d67-a491ef5461e1 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:24.341851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc10a31-9d71-4a01-8d67-a491ef5461e1 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:24.342141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc10a31-9d71-4a01-8d67-a491ef5461e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:24.360603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:24.369780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-dbc10a31-9d71-4a01-8d67-a491ef5461e1 None None] MFA Rules not processed for user `09c0fab0073545d48e6bf7755b5d4f18`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:24.371039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-dbc10a31-9d71-4a01-8d67-a491ef5461e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:24.435209 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1183/2359] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:24.439666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a480a4f-dae1-4371-abc9-402e84313cb9 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:24.500628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a480a4f-dae1-4371-abc9-402e84313cb9 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:24.505607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7b818-004a-4573-ae46-f2c39ed9e137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:24.506428 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7b818-004a-4573-ae46-f2c39ed9e137 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:24.506757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7b818-004a-4573-ae46-f2c39ed9e137 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:24.507096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7b818-004a-4573-ae46-f2c39ed9e137 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:24.507884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7b818-004a-4573-ae46-f2c39ed9e137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:24.509842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7b818-004a-4573-ae46-f2c39ed9e137 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:24.619752 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1177/2360] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:24 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:24.624008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc10a31-9d71-4a01-8d67-a491ef5461e1 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:24.634677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc10a31-9d71-4a01-8d67-a491ef5461e1 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:24.691026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd0cfa-40dc-442f-85fe-2951ed6afb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:24.691663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd0cfa-40dc-442f-85fe-2951ed6afb76 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:24.691946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd0cfa-40dc-442f-85fe-2951ed6afb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:24.692216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd0cfa-40dc-442f-85fe-2951ed6afb76 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:24.692916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd0cfa-40dc-442f-85fe-2951ed6afb76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:24.694895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd0cfa-40dc-442f-85fe-2951ed6afb76 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:24.790223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1184/2361] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:24 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:24.794269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7b818-004a-4573-ae46-f2c39ed9e137 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:24.805127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7b818-004a-4573-ae46-f2c39ed9e137 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:24.809665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20ef6c-6516-4487-8698-b78ffe9d5c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:24.810357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20ef6c-6516-4487-8698-b78ffe9d5c67 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:24.810678 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20ef6c-6516-4487-8698-b78ffe9d5c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:24.810954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20ef6c-6516-4487-8698-b78ffe9d5c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:24.811674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20ef6c-6516-4487-8698-b78ffe9d5c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:24.813272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20ef6c-6516-4487-8698-b78ffe9d5c67 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:24.823652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1178/2362] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:24.827735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd0cfa-40dc-442f-85fe-2951ed6afb76 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:24.838820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd0cfa-40dc-442f-85fe-2951ed6afb76 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:24.843537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44869e4c-c17a-43d0-894c-4c798c3d4d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:24.847863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44869e4c-c17a-43d0-894c-4c798c3d4d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:24.847863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44869e4c-c17a-43d0-894c-4c798c3d4d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:24.847863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44869e4c-c17a-43d0-894c-4c798c3d4d4b admin admin] PATH_INFO: `/v3/projects/e632796cbb6c4a2eb99ecff6700748a5/users/6c60f59eff7f472c924404cb1d9846ee/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:24.860418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44869e4c-c17a-43d0-894c-4c798c3d4d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e632796cbb6c4a2eb99ecff6700748a5, user_id=6c60f59eff7f472c924404cb1d9846ee, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:24.863132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44869e4c-c17a-43d0-894c-4c798c3d4d4b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:24.948695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1185/2363] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:24 2022] PUT /identity/v3/projects/e632796cbb6c4a2eb99ecff6700748a5/users/6c60f59eff7f472c924404cb1d9846ee/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:24.952665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20ef6c-6516-4487-8698-b78ffe9d5c67 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:24.963037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20ef6c-6516-4487-8698-b78ffe9d5c67 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:25.017598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d2cf1-e0a0-4aee-bd70-f71d0edf885d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:25.018191 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d2cf1-e0a0-4aee-bd70-f71d0edf885d admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:25.018495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d2cf1-e0a0-4aee-bd70-f71d0edf885d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:25.018786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d2cf1-e0a0-4aee-bd70-f71d0edf885d admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:25.020063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d2cf1-e0a0-4aee-bd70-f71d0edf885d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:25.022499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d2cf1-e0a0-4aee-bd70-f71d0edf885d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:25.033006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1179/2364] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:05:25.037752 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44869e4c-c17a-43d0-894c-4c798c3d4d4b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:25.048457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44869e4c-c17a-43d0-894c-4c798c3d4d4b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:25.053193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445a417-9da6-4905-8e9d-d7072f766c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:25.054074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445a417-9da6-4905-8e9d-d7072f766c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:25.054356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445a417-9da6-4905-8e9d-d7072f766c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:25.054741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445a417-9da6-4905-8e9d-d7072f766c31 admin admin] PATH_INFO: `/v3/projects/e632796cbb6c4a2eb99ecff6700748a5/users/6c60f59eff7f472c924404cb1d9846ee/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:25.056701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445a417-9da6-4905-8e9d-d7072f766c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e632796cbb6c4a2eb99ecff6700748a5, user_id=6c60f59eff7f472c924404cb1d9846ee, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:25.059360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445a417-9da6-4905-8e9d-d7072f766c31 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:25.070384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1186/2365] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:25 2022] PUT /identity/v3/projects/e632796cbb6c4a2eb99ecff6700748a5/users/6c60f59eff7f472c924404cb1d9846ee/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:25.080362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919e16a-2f5e-4781-9f5b-555e9958db2c None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:25.080670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919e16a-2f5e-4781-9f5b-555e9958db2c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:25.080948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919e16a-2f5e-4781-9f5b-555e9958db2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:25.106660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e919e16a-2f5e-4781-9f5b-555e9958db2c None None] MFA Rules not processed for user `6c60f59eff7f472c924404cb1d9846ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:25.107854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e919e16a-2f5e-4781-9f5b-555e9958db2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:25.151762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:25.174538 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1180/2366] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:25.185166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445a417-9da6-4905-8e9d-d7072f766c31 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:25.187552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6445a417-9da6-4905-8e9d-d7072f766c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:25.243554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445a417-9da6-4905-8e9d-d7072f766c31 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:25.247220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a119e79-bea8-470b-ba6a-da39a168e660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:25.247869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a119e79-bea8-470b-ba6a-da39a168e660 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:25.248154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a119e79-bea8-470b-ba6a-da39a168e660 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:25.248435 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a119e79-bea8-470b-ba6a-da39a168e660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:25.254865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:25.258353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a119e79-bea8-470b-ba6a-da39a168e660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:25.259908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a119e79-bea8-470b-ba6a-da39a168e660 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:25.269904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1187/2367] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:25.865756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb331372-c49b-4170-894a-90bdb106151c None None] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:25.866228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb331372-c49b-4170-894a-90bdb106151c None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:25.866379 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb331372-c49b-4170-894a-90bdb106151c None None] PATH_INFO: `/` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:25.868128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1181/2368] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 06:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:05:25.880781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc88d81-b4dd-4274-83e5-3158558f8d7b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:25.886637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc88d81-b4dd-4274-83e5-3158558f8d7b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:25.891720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc88d81-b4dd-4274-83e5-3158558f8d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:25.919223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ddc88d81-b4dd-4274-83e5-3158558f8d7b None None] MFA Rules not processed for user `ecd61347f8874b89aada19772ec56179`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:25.920381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ddc88d81-b4dd-4274-83e5-3158558f8d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:25.985867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1188/2369] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 06:05:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:26.000547 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb331372-c49b-4170-894a-90bdb106151c None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:26.006089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-eb331372-c49b-4170-894a-90bdb106151c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:26.025278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb331372-c49b-4170-894a-90bdb106151c None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:26.030454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c8e89-d7c5-44ee-bce4-763460d3e715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:26.031133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c8e89-d7c5-44ee-bce4-763460d3e715 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:26.031418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c8e89-d7c5-44ee-bce4-763460d3e715 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:26.031696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c8e89-d7c5-44ee-bce4-763460d3e715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:26.040742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c8e89-d7c5-44ee-bce4-763460d3e715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:26.042185 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c8e89-d7c5-44ee-bce4-763460d3e715 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:26.051344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1182/2370] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:26.137641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2da9d7-24eb-434b-95f8-721183685a92 None None] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:26.138203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2da9d7-24eb-434b-95f8-721183685a92 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:26.138301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2da9d7-24eb-434b-95f8-721183685a92 None None] PATH_INFO: `/` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:26.139729 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1189/2371] 10.222.0.17 () {54 vars in 921 bytes} [Mon Jul 25 06:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:05:26.148626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded3e3-b4a6-47a2-816f-965c6e954d9e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:26.148999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded3e3-b4a6-47a2-816f-965c6e954d9e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:26.149346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded3e3-b4a6-47a2-816f-965c6e954d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:26.168572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-35ded3e3-b4a6-47a2-816f-965c6e954d9e None None] MFA Rules not processed for user `067b827625e0427294b93b027597a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:26.170153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-35ded3e3-b4a6-47a2-816f-965c6e954d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:26.251981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1183/2372] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Jul 25 06:05:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:26.262913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2da9d7-24eb-434b-95f8-721183685a92 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:26.277115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2da9d7-24eb-434b-95f8-721183685a92 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:26.285288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a25d8b-0ca9-4432-960c-6135b531c642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:26.286277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a25d8b-0ca9-4432-960c-6135b531c642 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:26.286749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a25d8b-0ca9-4432-960c-6135b531c642 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:26.287222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a25d8b-0ca9-4432-960c-6135b531c642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:26.300387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a25d8b-0ca9-4432-960c-6135b531c642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:26.307361 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a25d8b-0ca9-4432-960c-6135b531c642 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:26.317414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1190/2373] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:26.395927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ded3e3-b4a6-47a2-816f-965c6e954d9e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:26.397331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-35ded3e3-b4a6-47a2-816f-965c6e954d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:26.408481 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ded3e3-b4a6-47a2-816f-965c6e954d9e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:26.412147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89f66-94dd-48e0-9ede-5f584dfdf3f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:26.412787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89f66-94dd-48e0-9ede-5f584dfdf3f5 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:26.413069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89f66-94dd-48e0-9ede-5f584dfdf3f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:26.413346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89f66-94dd-48e0-9ede-5f584dfdf3f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:26.489317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d89f66-94dd-48e0-9ede-5f584dfdf3f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:26.490832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d89f66-94dd-48e0-9ede-5f584dfdf3f5 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:26.509194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1184/2374] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:30.259620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db8078-7794-4d53-a064-c71dd5844656 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.260475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db8078-7794-4d53-a064-c71dd5844656 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.260814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db8078-7794-4d53-a064-c71dd5844656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.280346 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e6db8078-7794-4d53-a064-c71dd5844656 None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:30.281563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e6db8078-7794-4d53-a064-c71dd5844656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:30.349422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1191/2375] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 06:05:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:30.354379 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89f66-94dd-48e0-9ede-5f584dfdf3f5 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:30.355775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d1d89f66-94dd-48e0-9ede-5f584dfdf3f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:30.422217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:30.428143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89f66-94dd-48e0-9ede-5f584dfdf3f5 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:30.433224 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca4c14-53b4-401f-bc98-198f0127ebb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:30.433867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca4c14-53b4-401f-bc98-198f0127ebb4 service swift] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.434147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca4c14-53b4-401f-bc98-198f0127ebb4 service swift] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.434424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca4c14-53b4-401f-bc98-198f0127ebb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.442511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca4c14-53b4-401f-bc98-198f0127ebb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:30.444777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca4c14-53b4-401f-bc98-198f0127ebb4 service swift] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:30.452610 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1185/2376] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:05:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:05:32.532246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db8078-7794-4d53-a064-c71dd5844656 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:32.541912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db8078-7794-4d53-a064-c71dd5844656 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:32.545982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78be440-60a0-420a-b056-a478a0ddb426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:32.546726 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78be440-60a0-420a-b056-a478a0ddb426 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:32.547090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78be440-60a0-420a-b056-a478a0ddb426 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:32.547445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78be440-60a0-420a-b056-a478a0ddb426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:32.605339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78be440-60a0-420a-b056-a478a0ddb426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:32.606825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78be440-60a0-420a-b056-a478a0ddb426 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:32.617011 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1192/2377] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:34.372324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca4c14-53b4-401f-bc98-198f0127ebb4 service swift] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:34.373854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-bdca4c14-53b4-401f-bc98-198f0127ebb4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:34.385682 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca4c14-53b4-401f-bc98-198f0127ebb4 service swift] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:34.389290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe5543-34b7-4b13-bdcd-8f685f3b9caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:34.389927 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe5543-34b7-4b13-bdcd-8f685f3b9caf service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:34.390212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe5543-34b7-4b13-bdcd-8f685f3b9caf service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:34.390511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe5543-34b7-4b13-bdcd-8f685f3b9caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:34.392279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e1fe5543-34b7-4b13-bdcd-8f685f3b9caf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:34.457212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe5543-34b7-4b13-bdcd-8f685f3b9caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:34.458585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe5543-34b7-4b13-bdcd-8f685f3b9caf service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:34.468176 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1186/2378] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:36.636345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78be440-60a0-420a-b056-a478a0ddb426 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:36.649690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78be440-60a0-420a-b056-a478a0ddb426 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:36.668412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:36.669698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:36.670048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:36.671115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] PATH_INFO: `/v3/users/49b5ff6d18a945a78233ec48065bb60d` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:36.672224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b5ff6d18a945a78233ec48065bb60d)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:36.674351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:36.777238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:36.887775 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b5ff6d18a945a78233ec48065bb60d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:36.901583 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b5ff6d18a945a78233ec48065bb60d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:36.902307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b5ff6d18a945a78233ec48065bb60d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:37.003537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b5ff6d18a945a78233ec48065bb60d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:37.004374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b5ff6d18a945a78233ec48065bb60d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:37.023282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1193/2379] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:36 2022] DELETE /identity/v3/users/49b5ff6d18a945a78233ec48065bb60d => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:37.028016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b50f-5c11-493f-a507-082375c32f10 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:37.029036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b50f-5c11-493f-a507-082375c32f10 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:37.030312 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b50f-5c11-493f-a507-082375c32f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:37.048045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-ddd6b50f-5c11-493f-a507-082375c32f10 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:37.049320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ddd6b50f-5c11-493f-a507-082375c32f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:37.116030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1187/2380] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:37.126203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:37.127751 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:37.185771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a4ff9-a5f0-475b-827d-bbeaa6b0112c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:37.189341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa4dd5-92fe-43c9-8ad1-9edf38f23cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:37.189984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa4dd5-92fe-43c9-8ad1-9edf38f23cbf service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:37.190264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa4dd5-92fe-43c9-8ad1-9edf38f23cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:37.190572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa4dd5-92fe-43c9-8ad1-9edf38f23cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:37.200195 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa4dd5-92fe-43c9-8ad1-9edf38f23cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:37.201570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa4dd5-92fe-43c9-8ad1-9edf38f23cbf service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:37.210943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1194/2381] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:37.647338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6b50f-5c11-493f-a507-082375c32f10 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:37.649315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ddd6b50f-5c11-493f-a507-082375c32f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:37.668184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6b50f-5c11-493f-a507-082375c32f10 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:37.674145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:37.675299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:37.675918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:37.676279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] PATH_INFO: `/v3/projects/384c77590a9d4e64977bacaccce929ae` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:37.678052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=384c77590a9d4e64977bacaccce929ae)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:37.681948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:37.848081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384c77590a9d4e64977bacaccce929ae', 'request_id': 'req-7c22e679-323f-4a86-9c46-30684343bcbd'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:37.860147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384c77590a9d4e64977bacaccce929ae', 'request_id': 'req-7c22e679-323f-4a86-9c46-30684343bcbd'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:37.869413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384c77590a9d4e64977bacaccce929ae', 'request_id': 'req-7c22e679-323f-4a86-9c46-30684343bcbd'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:37.965252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] The token cache is being invalidate because project 384c77590a9d4e64977bacaccce929ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:37.965543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:37.966601 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:37.969245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1188/2382] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:37 2022] DELETE /identity/v3/projects/384c77590a9d4e64977bacaccce929ae => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:37.973023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa4dd5-92fe-43c9-8ad1-9edf38f23cbf service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:37.988051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa4dd5-92fe-43c9-8ad1-9edf38f23cbf service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:38.081210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:38.083733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:38.083733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:38.083733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] PATH_INFO: `/v3/users/09c0fab0073545d48e6bf7755b5d4f18` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:38.084035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c0fab0073545d48e6bf7755b5d4f18)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:38.086353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:38.292376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:38.293553 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c0fab0073545d48e6bf7755b5d4f18'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:38.294930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:38.296426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:38.302172 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c0fab0073545d48e6bf7755b5d4f18'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:38.305597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c0fab0073545d48e6bf7755b5d4f18'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:38.306821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:38.367893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22e679-323f-4a86-9c46-30684343bcbd admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:38.373141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b43d7-09d4-4a33-a8d2-1a094e0a6236 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:38.373796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b43d7-09d4-4a33-a8d2-1a094e0a6236 service placement] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:38.374082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b43d7-09d4-4a33-a8d2-1a094e0a6236 service placement] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:38.374361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b43d7-09d4-4a33-a8d2-1a094e0a6236 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:38.376173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-260b43d7-09d4-4a33-a8d2-1a094e0a6236 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:38.386529 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c0fab0073545d48e6bf7755b5d4f18'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:38.387162 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c0fab0073545d48e6bf7755b5d4f18'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:38.388077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b43d7-09d4-4a33-a8d2-1a094e0a6236 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:38.389683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b43d7-09d4-4a33-a8d2-1a094e0a6236 service placement] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:38.405532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1189/2383] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:05:38.405711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1195/2384] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:37 2022] DELETE /identity/v3/users/09c0fab0073545d48e6bf7755b5d4f18 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:38.451466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b43d7-09d4-4a33-a8d2-1a094e0a6236 service placement] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:38.453022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-260b43d7-09d4-4a33-a8d2-1a094e0a6236 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:38.467915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b43d7-09d4-4a33-a8d2-1a094e0a6236 service placement] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:38.525159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:38.526065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:38.526345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:38.526652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] PATH_INFO: `/v3/users/dc8b3ecc32c14e31b55156586129a40c` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:38.527985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc8b3ecc32c14e31b55156586129a40c)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:38.530025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:38.705574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc8b3ecc32c14e31b55156586129a40c'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:38.713840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc8b3ecc32c14e31b55156586129a40c'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:38.807557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc8b3ecc32c14e31b55156586129a40c'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:38.807557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc8b3ecc32c14e31b55156586129a40c'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:38.807557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc8b3ecc32c14e31b55156586129a40c'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:38.825164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1190/2385] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:38 2022] DELETE /identity/v3/users/dc8b3ecc32c14e31b55156586129a40c => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:38.919226 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:38.920579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:38.934808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a23e63-0873-4d57-b7d2-01ad5f49db59 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:39.008001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:39.009012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:39.009316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:39.009598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] PATH_INFO: `/v3/projects/d83e38aa761d414f8091b241dd6776fd` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:39.010925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d83e38aa761d414f8091b241dd6776fd)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:39.013023 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:39.098287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:39.100115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:39.122373 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd75f5-db08-4e20-a5c7-f69295a6ac9b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:39.139548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:39.160481 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd83e38aa761d414f8091b241dd6776fd', 'request_id': 'req-21f22cbd-6aaa-442a-9610-745a2252155a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.174335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd83e38aa761d414f8091b241dd6776fd', 'request_id': 'req-21f22cbd-6aaa-442a-9610-745a2252155a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.183567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd83e38aa761d414f8091b241dd6776fd', 'request_id': 'req-21f22cbd-6aaa-442a-9610-745a2252155a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.214695 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:39.215789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:39.216230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:39.216680 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] PATH_INFO: `/v3/projects/15678135086b4a5c83eefece6cb8f79a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:39.218324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15678135086b4a5c83eefece6cb8f79a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:39.220460 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:39.259982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] The token cache is being invalidate because project d83e38aa761d414f8091b241dd6776fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:39.260272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.264020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.264020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1196/2386] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:38 2022] DELETE /identity/v3/projects/d83e38aa761d414f8091b241dd6776fd => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:39.266574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:39.267931 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:39.281717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f22cbd-6aaa-442a-9610-745a2252155a admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:39.286455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:39.287344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:39.287623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:39.288104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] PATH_INFO: `/v3/users/6c60f59eff7f472c924404cb1d9846ee` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:39.289710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c60f59eff7f472c924404cb1d9846ee)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:39.291642 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:39.352870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15678135086b4a5c83eefece6cb8f79a', 'request_id': 'req-4db0323e-1e07-45cc-8c6c-1f8f30434e62'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.359180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15678135086b4a5c83eefece6cb8f79a', 'request_id': 'req-4db0323e-1e07-45cc-8c6c-1f8f30434e62'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.364594 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15678135086b4a5c83eefece6cb8f79a', 'request_id': 'req-4db0323e-1e07-45cc-8c6c-1f8f30434e62'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.454547 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] The token cache is being invalidate because project 15678135086b4a5c83eefece6cb8f79a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:39.454853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.455469 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4db0323e-1e07-45cc-8c6c-1f8f30434e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.457016 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1191/2387] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:39 2022] DELETE /identity/v3/projects/15678135086b4a5c83eefece6cb8f79a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:39.462459 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601240f1-7590-498b-88fd-28ccc70a49fb None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:39.462775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601240f1-7590-498b-88fd-28ccc70a49fb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:39.463053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601240f1-7590-498b-88fd-28ccc70a49fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:39.482979 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-601240f1-7590-498b-88fd-28ccc70a49fb None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:39.484212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-601240f1-7590-498b-88fd-28ccc70a49fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:39.487282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c60f59eff7f472c924404cb1d9846ee'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.491964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:39.495007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c60f59eff7f472c924404cb1d9846ee'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.495800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c60f59eff7f472c924404cb1d9846ee'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.550685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1192/2388] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:39.556555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad79853-f30d-46a3-ba2c-d9fdedb1a534 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:39.556850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad79853-f30d-46a3-ba2c-d9fdedb1a534 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:39.557136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad79853-f30d-46a3-ba2c-d9fdedb1a534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:39.576301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-2ad79853-f30d-46a3-ba2c-d9fdedb1a534 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:39.577571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2ad79853-f30d-46a3-ba2c-d9fdedb1a534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:39.587625 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1193/2389] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:39.591651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad79853-f30d-46a3-ba2c-d9fdedb1a534 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:39.597589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c60f59eff7f472c924404cb1d9846ee'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.598322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c60f59eff7f472c924404cb1d9846ee'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:39.600926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad79853-f30d-46a3-ba2c-d9fdedb1a534 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:39.608149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff928087-f1d4-485b-a614-473a254b6e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:39.608149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff928087-f1d4-485b-a614-473a254b6e60 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:39.608149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff928087-f1d4-485b-a614-473a254b6e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:39.608149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff928087-f1d4-485b-a614-473a254b6e60 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:39.608578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff928087-f1d4-485b-a614-473a254b6e60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:39.608578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff928087-f1d4-485b-a614-473a254b6e60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:39.611787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff928087-f1d4-485b-a614-473a254b6e60 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:39.622337 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1194/2390] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:05:39.626831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff928087-f1d4-485b-a614-473a254b6e60 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:39.628790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ff928087-f1d4-485b-a614-473a254b6e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:39.635204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1197/2391] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:39 2022] DELETE /identity/v3/users/6c60f59eff7f472c924404cb1d9846ee => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:39.643447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff928087-f1d4-485b-a614-473a254b6e60 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:39.708124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7711de9-8780-47cf-95a9-d3dbcafffc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:39.708818 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7711de9-8780-47cf-95a9-d3dbcafffc1c admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:39.709105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7711de9-8780-47cf-95a9-d3dbcafffc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:39.709383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7711de9-8780-47cf-95a9-d3dbcafffc1c admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:39.710100 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7711de9-8780-47cf-95a9-d3dbcafffc1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:39.712631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7711de9-8780-47cf-95a9-d3dbcafffc1c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:39.885141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1195/2392] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:39 2022] POST /identity/v3/projects => generated 385 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:39.891536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:39.902756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552551ee-3f69-464c-bbf2-5268038096b4 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:39.970817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cfc61-3cf1-450c-ac4e-a25dba4e5890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:39.971502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cfc61-3cf1-450c-ac4e-a25dba4e5890 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:39.971792 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cfc61-3cf1-450c-ac4e-a25dba4e5890 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:39.972069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cfc61-3cf1-450c-ac4e-a25dba4e5890 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:39.972781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cfc61-3cf1-450c-ac4e-a25dba4e5890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:39.974733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cfc61-3cf1-450c-ac4e-a25dba4e5890 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:39.981030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:40.066636 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1198/2393] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:39 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:40.071467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7711de9-8780-47cf-95a9-d3dbcafffc1c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.081757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7711de9-8780-47cf-95a9-d3dbcafffc1c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.086385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ae612-5f59-4ad8-8527-be11ea8e129e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.087067 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ae612-5f59-4ad8-8527-be11ea8e129e admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.087346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ae612-5f59-4ad8-8527-be11ea8e129e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.087616 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ae612-5f59-4ad8-8527-be11ea8e129e admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.088398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ae612-5f59-4ad8-8527-be11ea8e129e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.090006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ae612-5f59-4ad8-8527-be11ea8e129e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.099672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1196/2394] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:40.107953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cfc61-3cf1-450c-ac4e-a25dba4e5890 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.118622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cfc61-3cf1-450c-ac4e-a25dba4e5890 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.123226 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1253d4-2a7f-457d-ac6f-912aae82197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.123943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1253d4-2a7f-457d-ac6f-912aae82197c admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.124475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1253d4-2a7f-457d-ac6f-912aae82197c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.124949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1253d4-2a7f-457d-ac6f-912aae82197c admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.126051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1253d4-2a7f-457d-ac6f-912aae82197c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.127730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1253d4-2a7f-457d-ac6f-912aae82197c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.138424 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1199/2395] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:40.142953 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ae612-5f59-4ad8-8527-be11ea8e129e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.152978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ae612-5f59-4ad8-8527-be11ea8e129e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.162923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87ac54-dfaf-4758-ab20-9144494ab5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.162923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87ac54-dfaf-4758-ab20-9144494ab5af admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.162923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87ac54-dfaf-4758-ab20-9144494ab5af admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.162923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87ac54-dfaf-4758-ab20-9144494ab5af admin admin] PATH_INFO: `/v3/projects/3b3a41b96ab345198e7725662e13a472/users/c9f34fe2abc74bf0ae540a3d99a8e807/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.165615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1253d4-2a7f-457d-ac6f-912aae82197c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.167466 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-cf1253d4-2a7f-457d-ac6f-912aae82197c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:40.180065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87ac54-dfaf-4758-ab20-9144494ab5af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3a41b96ab345198e7725662e13a472, user_id=c9f34fe2abc74bf0ae540a3d99a8e807, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.182922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87ac54-dfaf-4758-ab20-9144494ab5af admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.192106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1253d4-2a7f-457d-ac6f-912aae82197c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.195104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.196485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.196791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.197098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] PATH_INFO: `/v3/projects/e632796cbb6c4a2eb99ecff6700748a5` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.198250 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e632796cbb6c4a2eb99ecff6700748a5)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.200195 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.273789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1197/2396] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:40 2022] PUT /identity/v3/projects/3b3a41b96ab345198e7725662e13a472/users/c9f34fe2abc74bf0ae540a3d99a8e807/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:40.283979 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec1de5-f71a-4e93-bdc8-f94c1bcce290 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.284269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec1de5-f71a-4e93-bdc8-f94c1bcce290 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.284552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec1de5-f71a-4e93-bdc8-f94c1bcce290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.315817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-29ec1de5-f71a-4e93-bdc8-f94c1bcce290 None None] MFA Rules not processed for user `c9f34fe2abc74bf0ae540a3d99a8e807`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:40.317431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-29ec1de5-f71a-4e93-bdc8-f94c1bcce290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:40.330256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e632796cbb6c4a2eb99ecff6700748a5', 'request_id': 'req-2e4ba454-0f01-44b9-b2a9-fa333cbea404'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:40.339007 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e632796cbb6c4a2eb99ecff6700748a5', 'request_id': 'req-2e4ba454-0f01-44b9-b2a9-fa333cbea404'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:40.345467 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e632796cbb6c4a2eb99ecff6700748a5', 'request_id': 'req-2e4ba454-0f01-44b9-b2a9-fa333cbea404'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:40.388160 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1198/2397] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:40.391786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec1de5-f71a-4e93-bdc8-f94c1bcce290 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.443603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] The token cache is being invalidate because project e632796cbb6c4a2eb99ecff6700748a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:40.443884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:40.444525 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2e4ba454-0f01-44b9-b2a9-fa333cbea404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:40.446426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1200/2398] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:40 2022] DELETE /identity/v3/projects/e632796cbb6c4a2eb99ecff6700748a5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:40.452159 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94aca9-f98f-4590-93c5-2a4ea44b1373 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.452482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94aca9-f98f-4590-93c5-2a4ea44b1373 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.452835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94aca9-f98f-4590-93c5-2a4ea44b1373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.459729 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec1de5-f71a-4e93-bdc8-f94c1bcce290 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.467058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494be9d-8626-43f3-906e-a72d8aef9bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.467777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494be9d-8626-43f3-906e-a72d8aef9bfa admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.469321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494be9d-8626-43f3-906e-a72d8aef9bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.469321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494be9d-8626-43f3-906e-a72d8aef9bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.469321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494be9d-8626-43f3-906e-a72d8aef9bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.471053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494be9d-8626-43f3-906e-a72d8aef9bfa admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.472621 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2f94aca9-f98f-4590-93c5-2a4ea44b1373 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:40.473828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2f94aca9-f98f-4590-93c5-2a4ea44b1373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:40.482430 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1201/2399] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:40.490841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44282e09-b0a4-4613-a423-402c7817f1bc None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.491136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44282e09-b0a4-4613-a423-402c7817f1bc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.491436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44282e09-b0a4-4613-a423-402c7817f1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.509533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-44282e09-b0a4-4613-a423-402c7817f1bc None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:40.510768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-44282e09-b0a4-4613-a423-402c7817f1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:40.518728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1202/2400] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:40.522781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44282e09-b0a4-4613-a423-402c7817f1bc None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.531518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44282e09-b0a4-4613-a423-402c7817f1bc None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.536180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec6a4f-e304-438c-aa30-4665cc15123c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.536829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec6a4f-e304-438c-aa30-4665cc15123c admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.537114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec6a4f-e304-438c-aa30-4665cc15123c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.537393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec6a4f-e304-438c-aa30-4665cc15123c admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.538102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec6a4f-e304-438c-aa30-4665cc15123c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:40.538414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec6a4f-e304-438c-aa30-4665cc15123c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.540566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec6a4f-e304-438c-aa30-4665cc15123c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.552017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1203/2401] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:05:40.556614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec6a4f-e304-438c-aa30-4665cc15123c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.562337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:40.566224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec6a4f-e304-438c-aa30-4665cc15123c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.571096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1199/2402] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:40 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:40.571639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd373be9-0392-405a-898e-277555e657a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.572281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd373be9-0392-405a-898e-277555e657a8 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.572590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd373be9-0392-405a-898e-277555e657a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.572872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd373be9-0392-405a-898e-277555e657a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.573557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd373be9-0392-405a-898e-277555e657a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.575371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494be9d-8626-43f3-906e-a72d8aef9bfa admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.575491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd373be9-0392-405a-898e-277555e657a8 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.576967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2494be9d-8626-43f3-906e-a72d8aef9bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:40.639988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494be9d-8626-43f3-906e-a72d8aef9bfa admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.644642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd286626-9d98-4527-bf53-1f4ff67068eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.645307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd286626-9d98-4527-bf53-1f4ff67068eb admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.645589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd286626-9d98-4527-bf53-1f4ff67068eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.645865 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd286626-9d98-4527-bf53-1f4ff67068eb admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.646590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd286626-9d98-4527-bf53-1f4ff67068eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.648497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd286626-9d98-4527-bf53-1f4ff67068eb admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.684207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1204/2403] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:40 2022] POST /identity/v3/projects => generated 391 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:40.685255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd373be9-0392-405a-898e-277555e657a8 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.750029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd373be9-0392-405a-898e-277555e657a8 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.754833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7715d27-69cd-47df-89d8-43981447f0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.755504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7715d27-69cd-47df-89d8-43981447f0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.755781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7715d27-69cd-47df-89d8-43981447f0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.756052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7715d27-69cd-47df-89d8-43981447f0c2 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.756735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7715d27-69cd-47df-89d8-43981447f0c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.759415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7715d27-69cd-47df-89d8-43981447f0c2 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.768629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1200/2404] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:40 2022] POST /identity/v3/users => generated 300 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:40.770038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd286626-9d98-4527-bf53-1f4ff67068eb admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.789925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd286626-9d98-4527-bf53-1f4ff67068eb admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.796543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6d054-7881-41db-8d94-a008b3ea5b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.797319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6d054-7881-41db-8d94-a008b3ea5b2e admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.797779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6d054-7881-41db-8d94-a008b3ea5b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.798053 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6d054-7881-41db-8d94-a008b3ea5b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.798907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6d054-7881-41db-8d94-a008b3ea5b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.801022 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6d054-7881-41db-8d94-a008b3ea5b2e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.812416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1201/2405] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:05:40.816992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6d054-7881-41db-8d94-a008b3ea5b2e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.827782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6d054-7881-41db-8d94-a008b3ea5b2e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.832969 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1f21d-41bb-4ea1-9911-8444ad9470c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.833709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1f21d-41bb-4ea1-9911-8444ad9470c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.833995 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1f21d-41bb-4ea1-9911-8444ad9470c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.834270 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1f21d-41bb-4ea1-9911-8444ad9470c7 admin admin] PATH_INFO: `/v3/projects/5272912e604b41e49011572101f6c784/users/a87cd5a9cab94bc2a0cef15e560d5062/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.850518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1f21d-41bb-4ea1-9911-8444ad9470c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5272912e604b41e49011572101f6c784, user_id=a87cd5a9cab94bc2a0cef15e560d5062, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.853958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1f21d-41bb-4ea1-9911-8444ad9470c7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.883704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1205/2406] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:40 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:40.884807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7715d27-69cd-47df-89d8-43981447f0c2 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.894817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7715d27-69cd-47df-89d8-43981447f0c2 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.900483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563576f-5287-4d7e-bdd9-19c8a9ff30b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.900846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563576f-5287-4d7e-bdd9-19c8a9ff30b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.901135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563576f-5287-4d7e-bdd9-19c8a9ff30b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.901451 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563576f-5287-4d7e-bdd9-19c8a9ff30b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.902666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563576f-5287-4d7e-bdd9-19c8a9ff30b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.903835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563576f-5287-4d7e-bdd9-19c8a9ff30b0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.904532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:40.915472 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1206/2407] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:40.923509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563576f-5287-4d7e-bdd9-19c8a9ff30b0 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.936386 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563576f-5287-4d7e-bdd9-19c8a9ff30b0 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:40.942141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c88ad-2561-4a3c-99a1-5ec460292364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:40.947309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c88ad-2561-4a3c-99a1-5ec460292364 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:40.947309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c88ad-2561-4a3c-99a1-5ec460292364 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:40.947309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c88ad-2561-4a3c-99a1-5ec460292364 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:40.947309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c88ad-2561-4a3c-99a1-5ec460292364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:40.947309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c88ad-2561-4a3c-99a1-5ec460292364 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:40.950996 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1202/2408] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:40 2022] PUT /identity/v3/projects/5272912e604b41e49011572101f6c784/users/a87cd5a9cab94bc2a0cef15e560d5062/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:40.952392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1f21d-41bb-4ea1-9911-8444ad9470c7 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.959192 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1207/2409] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:05:40.965508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c88ad-2561-4a3c-99a1-5ec460292364 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:40.982601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c88ad-2561-4a3c-99a1-5ec460292364 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.027676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1f21d-41bb-4ea1-9911-8444ad9470c7 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.032302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088d988-cb43-44ef-8c97-a9baef975e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.033560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d988-cb43-44ef-8c97-a9baef975e98 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.034054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d988-cb43-44ef-8c97-a9baef975e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.034607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088d988-cb43-44ef-8c97-a9baef975e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.035374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088d988-cb43-44ef-8c97-a9baef975e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.037302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088d988-cb43-44ef-8c97-a9baef975e98 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.043354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bff458-275d-45be-bd23-810c8959eaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.044264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bff458-275d-45be-bd23-810c8959eaa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.044555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bff458-275d-45be-bd23-810c8959eaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.044836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bff458-275d-45be-bd23-810c8959eaa1 admin admin] PATH_INFO: `/v3/projects/47d89a55c3b2492c9aa64eb2205e5a0d/users/6a7d79f89cde4565b504056ed8a90637/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.050259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1203/2410] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:41.054989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088d988-cb43-44ef-8c97-a9baef975e98 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.061450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bff458-275d-45be-bd23-810c8959eaa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d89a55c3b2492c9aa64eb2205e5a0d, user_id=6a7d79f89cde4565b504056ed8a90637, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.067941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bff458-275d-45be-bd23-810c8959eaa1 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.068593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088d988-cb43-44ef-8c97-a9baef975e98 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.075177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f129d-8ca0-4a2a-b235-7e2f877f1687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.076753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f129d-8ca0-4a2a-b235-7e2f877f1687 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.077042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f129d-8ca0-4a2a-b235-7e2f877f1687 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.077319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f129d-8ca0-4a2a-b235-7e2f877f1687 admin admin] PATH_INFO: `/v3/projects/5272912e604b41e49011572101f6c784/users/a87cd5a9cab94bc2a0cef15e560d5062/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.082960 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f129d-8ca0-4a2a-b235-7e2f877f1687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5272912e604b41e49011572101f6c784, user_id=a87cd5a9cab94bc2a0cef15e560d5062, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.086808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f129d-8ca0-4a2a-b235-7e2f877f1687 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.103001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1204/2411] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:41 2022] PUT /identity/v3/projects/5272912e604b41e49011572101f6c784/users/a87cd5a9cab94bc2a0cef15e560d5062/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:41.114266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a79c9b-0069-429e-a8e5-fb1eb6440a06 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.114625 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a79c9b-0069-429e-a8e5-fb1eb6440a06 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.114903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a79c9b-0069-429e-a8e5-fb1eb6440a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.153747 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a8a79c9b-0069-429e-a8e5-fb1eb6440a06 None None] MFA Rules not processed for user `a87cd5a9cab94bc2a0cef15e560d5062`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:41.154578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1208/2412] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:40 2022] PUT /identity/v3/projects/47d89a55c3b2492c9aa64eb2205e5a0d/users/6a7d79f89cde4565b504056ed8a90637/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:41.155973 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a8a79c9b-0069-429e-a8e5-fb1eb6440a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:41.165748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cf703-0bce-4047-9b1e-3eb09c2ca0e7 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.166070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cf703-0bce-4047-9b1e-3eb09c2ca0e7 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.166309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752cf703-0bce-4047-9b1e-3eb09c2ca0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.206559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-752cf703-0bce-4047-9b1e-3eb09c2ca0e7 None None] MFA Rules not processed for user `6a7d79f89cde4565b504056ed8a90637`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:41.207997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-752cf703-0bce-4047-9b1e-3eb09c2ca0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:41.267232 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1205/2413] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:41.278414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:41.280629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a79c9b-0069-429e-a8e5-fb1eb6440a06 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.282104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a8a79c9b-0069-429e-a8e5-fb1eb6440a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:41.289537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1209/2414] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:41.294353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cf703-0bce-4047-9b1e-3eb09c2ca0e7 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.295976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:41.344312 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a79c9b-0069-429e-a8e5-fb1eb6440a06 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.347896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be182231-4742-4f08-8725-e77d2894b86b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.348545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be182231-4742-4f08-8725-e77d2894b86b service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.348825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be182231-4742-4f08-8725-e77d2894b86b service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.349099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be182231-4742-4f08-8725-e77d2894b86b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.358266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be182231-4742-4f08-8725-e77d2894b86b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.359737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be182231-4742-4f08-8725-e77d2894b86b service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.360085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752cf703-0bce-4047-9b1e-3eb09c2ca0e7 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.364879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39d544-2255-45f4-9c28-b6367e48da24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.365527 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39d544-2255-45f4-9c28-b6367e48da24 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.365833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39d544-2255-45f4-9c28-b6367e48da24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.366081 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39d544-2255-45f4-9c28-b6367e48da24 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.367253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39d544-2255-45f4-9c28-b6367e48da24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.368957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39d544-2255-45f4-9c28-b6367e48da24 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.369880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1206/2415] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:05:41.435675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be182231-4742-4f08-8725-e77d2894b86b service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.447351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be182231-4742-4f08-8725-e77d2894b86b service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.452041 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.452900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.453189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.453438 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] PATH_INFO: `/v3/users/c9f34fe2abc74bf0ae540a3d99a8e807` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.454677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f34fe2abc74bf0ae540a3d99a8e807)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.456590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.475094 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1210/2416] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:41 2022] POST /identity/v3/projects => generated 391 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:41.479115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39d544-2255-45f4-9c28-b6367e48da24 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.551938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39d544-2255-45f4-9c28-b6367e48da24 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.557401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39964f-c365-4595-8a67-9d7df9f8a4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.558075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39964f-c365-4595-8a67-9d7df9f8a4fe admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.558400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39964f-c365-4595-8a67-9d7df9f8a4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.558712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39964f-c365-4595-8a67-9d7df9f8a4fe admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.559387 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39964f-c365-4595-8a67-9d7df9f8a4fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.561356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39964f-c365-4595-8a67-9d7df9f8a4fe admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.630003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f34fe2abc74bf0ae540a3d99a8e807'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:41.637404 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f34fe2abc74bf0ae540a3d99a8e807'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:41.654017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1211/2417] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:41 2022] POST /identity/v3/users => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:41.655187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39964f-c365-4595-8a67-9d7df9f8a4fe admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.665829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39964f-c365-4595-8a67-9d7df9f8a4fe admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.671722 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4bf64-e029-4be9-b9e1-73ba419c6511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.672397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4bf64-e029-4be9-b9e1-73ba419c6511 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.672684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4bf64-e029-4be9-b9e1-73ba419c6511 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.672965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4bf64-e029-4be9-b9e1-73ba419c6511 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.673690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4bf64-e029-4be9-b9e1-73ba419c6511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.675342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4bf64-e029-4be9-b9e1-73ba419c6511 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.685697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1212/2418] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:05:41.689901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4bf64-e029-4be9-b9e1-73ba419c6511 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.699399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4bf64-e029-4be9-b9e1-73ba419c6511 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.703981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58febc-11f1-4b42-9c5d-c2297c203f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.705131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58febc-11f1-4b42-9c5d-c2297c203f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.705412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58febc-11f1-4b42-9c5d-c2297c203f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.705690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58febc-11f1-4b42-9c5d-c2297c203f89 admin admin] PATH_INFO: `/v3/projects/5079a0aea9e34ea2a51d01f0d1c65d91/users/14fadf4fdd344cd698d9d0fe5a06d25c/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.705794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f34fe2abc74bf0ae540a3d99a8e807'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:41.706447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f34fe2abc74bf0ae540a3d99a8e807'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:41.707127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2ad70c83-b4f5-447b-910d-a03e60bc942b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f34fe2abc74bf0ae540a3d99a8e807'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:41.721896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58febc-11f1-4b42-9c5d-c2297c203f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5079a0aea9e34ea2a51d01f0d1c65d91, user_id=14fadf4fdd344cd698d9d0fe5a06d25c, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.724627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58febc-11f1-4b42-9c5d-c2297c203f89 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.728424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1207/2419] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:41 2022] DELETE /identity/v3/users/c9f34fe2abc74bf0ae540a3d99a8e807 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:41.733264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa09820-a54c-422b-afe6-b74a5d730934 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.733264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa09820-a54c-422b-afe6-b74a5d730934 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.733264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa09820-a54c-422b-afe6-b74a5d730934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.750619 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-5aa09820-a54c-422b-afe6-b74a5d730934 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:41.751816 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5aa09820-a54c-422b-afe6-b74a5d730934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:41.775346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:41.810631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1208/2420] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:41.816318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1213/2421] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:41 2022] PUT /identity/v3/projects/5079a0aea9e34ea2a51d01f0d1c65d91/users/14fadf4fdd344cd698d9d0fe5a06d25c/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:41.820358 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa09820-a54c-422b-afe6-b74a5d730934 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.821753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5aa09820-a54c-422b-afe6-b74a5d730934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:41.821871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58febc-11f1-4b42-9c5d-c2297c203f89 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.823648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ef58febc-11f1-4b42-9c5d-c2297c203f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:41.828731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:41.879336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa09820-a54c-422b-afe6-b74a5d730934 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.888501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58febc-11f1-4b42-9c5d-c2297c203f89 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.893302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea31dd-583c-45a5-8970-75110de20716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.893983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea31dd-583c-45a5-8970-75110de20716 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.894273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea31dd-583c-45a5-8970-75110de20716 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.894693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea31dd-583c-45a5-8970-75110de20716 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.895864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea31dd-583c-45a5-8970-75110de20716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.897044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea31dd-583c-45a5-8970-75110de20716 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.908373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1214/2422] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:05:41.912419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea31dd-583c-45a5-8970-75110de20716 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.922720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea31dd-583c-45a5-8970-75110de20716 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.928085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac27e65-bd3e-4b7d-a93d-523762910a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.928966 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac27e65-bd3e-4b7d-a93d-523762910a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.929252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac27e65-bd3e-4b7d-a93d-523762910a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.929519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac27e65-bd3e-4b7d-a93d-523762910a75 admin admin] PATH_INFO: `/v3/projects/5079a0aea9e34ea2a51d01f0d1c65d91/users/14fadf4fdd344cd698d9d0fe5a06d25c/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.931532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac27e65-bd3e-4b7d-a93d-523762910a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5079a0aea9e34ea2a51d01f0d1c65d91, user_id=14fadf4fdd344cd698d9d0fe5a06d25c, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.934317 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac27e65-bd3e-4b7d-a93d-523762910a75 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.938032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32f837-35c2-44c6-8f78-112071c9e931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.938032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32f837-35c2-44c6-8f78-112071c9e931 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.938032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32f837-35c2-44c6-8f78-112071c9e931 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.938032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32f837-35c2-44c6-8f78-112071c9e931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.945897 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1215/2423] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:41 2022] PUT /identity/v3/projects/5079a0aea9e34ea2a51d01f0d1c65d91/users/14fadf4fdd344cd698d9d0fe5a06d25c/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:41.954630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32f837-35c2-44c6-8f78-112071c9e931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.954630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32f837-35c2-44c6-8f78-112071c9e931 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.956092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0586b17-a5f6-4f61-a7d5-d478005d1916 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.956373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0586b17-a5f6-4f61-a7d5-d478005d1916 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.956673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0586b17-a5f6-4f61-a7d5-d478005d1916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.966686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1209/2424] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:41.987932 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a0586b17-a5f6-4f61-a7d5-d478005d1916 None None] MFA Rules not processed for user `14fadf4fdd344cd698d9d0fe5a06d25c`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:41.989881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a0586b17-a5f6-4f61-a7d5-d478005d1916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:42.070787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1216/2425] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:42.081594 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32f837-35c2-44c6-8f78-112071c9e931 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:42.083033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3a32f837-35c2-44c6-8f78-112071c9e931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:42.139936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32f837-35c2-44c6-8f78-112071c9e931 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:42.185980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:42.196646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3a9f5-71dd-4425-8fe9-531287063f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:42.197328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3a9f5-71dd-4425-8fe9-531287063f50 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:42.197615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3a9f5-71dd-4425-8fe9-531287063f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:42.197901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3a9f5-71dd-4425-8fe9-531287063f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:42.208598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3a9f5-71dd-4425-8fe9-531287063f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:42.210056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b3a9f5-71dd-4425-8fe9-531287063f50 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:42.220596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1210/2426] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:42.268857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0586b17-a5f6-4f61-a7d5-d478005d1916 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:42.270433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a0586b17-a5f6-4f61-a7d5-d478005d1916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:42.282667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0586b17-a5f6-4f61-a7d5-d478005d1916 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:42.286304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485b8c4-0303-4ef8-ba22-982407f312a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:42.287053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485b8c4-0303-4ef8-ba22-982407f312a3 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:42.287425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485b8c4-0303-4ef8-ba22-982407f312a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:42.287772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485b8c4-0303-4ef8-ba22-982407f312a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:42.289695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a485b8c4-0303-4ef8-ba22-982407f312a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:42.350248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485b8c4-0303-4ef8-ba22-982407f312a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:42.351789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485b8c4-0303-4ef8-ba22-982407f312a3 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:42.360603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1217/2427] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:42.399412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3a9f5-71dd-4425-8fe9-531287063f50 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:42.399412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-34b3a9f5-71dd-4425-8fe9-531287063f50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:42.480651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3a9f5-71dd-4425-8fe9-531287063f50 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:42.485608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:42.486496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:42.486792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:42.487077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] PATH_INFO: `/v3/projects/3b3a41b96ab345198e7725662e13a472` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:42.488300 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3a41b96ab345198e7725662e13a472)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:42.490282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:42.605872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3a41b96ab345198e7725662e13a472', 'request_id': 'req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:42.612003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3a41b96ab345198e7725662e13a472', 'request_id': 'req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:42.617941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3a41b96ab345198e7725662e13a472', 'request_id': 'req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:42.708311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] The token cache is being invalidate because project 3b3a41b96ab345198e7725662e13a472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:42.709312 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:42.709981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:42.711603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1211/2428] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:42 2022] DELETE /identity/v3/projects/3b3a41b96ab345198e7725662e13a472 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:42.715763 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485b8c4-0303-4ef8-ba22-982407f312a3 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:42.727827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485b8c4-0303-4ef8-ba22-982407f312a3 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:42.733185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:42.797203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:42.798084 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:42.798367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:42.798671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] PATH_INFO: `/v3/users/a87cd5a9cab94bc2a0cef15e560d5062` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:42.799861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a87cd5a9cab94bc2a0cef15e560d5062)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:42.801773 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:43.072577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a87cd5a9cab94bc2a0cef15e560d5062'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:43.079862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a87cd5a9cab94bc2a0cef15e560d5062'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:43.080726 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a87cd5a9cab94bc2a0cef15e560d5062'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:43.152806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a87cd5a9cab94bc2a0cef15e560d5062'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:43.153474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-aba93607-51c6-4832-b2d0-b4f814a8e074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a87cd5a9cab94bc2a0cef15e560d5062'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:43.171342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1218/2429] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:42 2022] DELETE /identity/v3/users/a87cd5a9cab94bc2a0cef15e560d5062 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:43.611485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:43.612851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:43.628474 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84dfd4-a162-4066-b166-9a3ae7ebc2ff admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:43.691625 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:43.692559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:43.692845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:43.693133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] PATH_INFO: `/v3/projects/5272912e604b41e49011572101f6c784` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:43.694373 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5272912e604b41e49011572101f6c784)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:43.696886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:43.718439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:43.866635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5272912e604b41e49011572101f6c784', 'request_id': 'req-6091992e-332a-421e-b21d-6dac804b3b8d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:43.873636 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5272912e604b41e49011572101f6c784', 'request_id': 'req-6091992e-332a-421e-b21d-6dac804b3b8d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:43.878984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5272912e604b41e49011572101f6c784', 'request_id': 'req-6091992e-332a-421e-b21d-6dac804b3b8d'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:43.973570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] The token cache is being invalidate because project 5272912e604b41e49011572101f6c784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:43.973861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:43.974508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-6091992e-332a-421e-b21d-6dac804b3b8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:43.976054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1212/2430] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:43 2022] DELETE /identity/v3/projects/5272912e604b41e49011572101f6c784 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:43.981559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f63ed-9917-423f-bade-72e5ca080d1b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:43.981853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f63ed-9917-423f-bade-72e5ca080d1b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:43.984498 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f63ed-9917-423f-bade-72e5ca080d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:44.000425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-dd0f63ed-9917-423f-bade-72e5ca080d1b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:44.001612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-dd0f63ed-9917-423f-bade-72e5ca080d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:44.061750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1219/2431] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:44.069514 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd00b8-76ca-4155-be60-7d3522d2b284 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:44.069808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd00b8-76ca-4155-be60-7d3522d2b284 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:44.070091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd00b8-76ca-4155-be60-7d3522d2b284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:44.088762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4cfd00b8-76ca-4155-be60-7d3522d2b284 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:44.090010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4cfd00b8-76ca-4155-be60-7d3522d2b284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:44.098100 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1213/2432] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:44.103009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f63ed-9917-423f-bade-72e5ca080d1b None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:44.113046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f63ed-9917-423f-bade-72e5ca080d1b None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:44.117993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd3d5d-914a-4324-89a6-55fea96d9961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:44.118692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd3d5d-914a-4324-89a6-55fea96d9961 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:44.119043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd3d5d-914a-4324-89a6-55fea96d9961 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:44.121852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd3d5d-914a-4324-89a6-55fea96d9961 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:44.121852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd3d5d-914a-4324-89a6-55fea96d9961 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:44.121852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd3d5d-914a-4324-89a6-55fea96d9961 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:44.122153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd3d5d-914a-4324-89a6-55fea96d9961 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:44.134299 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1220/2433] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:05:44.138701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd00b8-76ca-4155-be60-7d3522d2b284 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:44.147837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd00b8-76ca-4155-be60-7d3522d2b284 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:44.153238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16083453-de4e-4956-91b7-80754589224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:44.153889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16083453-de4e-4956-91b7-80754589224f admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:44.154176 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16083453-de4e-4956-91b7-80754589224f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:44.154455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16083453-de4e-4956-91b7-80754589224f admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:44.155171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16083453-de4e-4956-91b7-80754589224f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:44.157168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16083453-de4e-4956-91b7-80754589224f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:44.281446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1214/2434] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:44 2022] POST /identity/v3/projects => generated 407 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:44.285700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd3d5d-914a-4324-89a6-55fea96d9961 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:44.295836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd3d5d-914a-4324-89a6-55fea96d9961 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:44.307701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:44.354720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a150f-9d6b-4a0b-b047-63c5cf9e3fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:44.355454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a150f-9d6b-4a0b-b047-63c5cf9e3fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:44.355735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a150f-9d6b-4a0b-b047-63c5cf9e3fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:44.356012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a150f-9d6b-4a0b-b047-63c5cf9e3fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:44.356748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a150f-9d6b-4a0b-b047-63c5cf9e3fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:44.358787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a150f-9d6b-4a0b-b047-63c5cf9e3fe5 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:44.449825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1221/2435] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:44 2022] POST /identity/v3/users => generated 304 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:44.454842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16083453-de4e-4956-91b7-80754589224f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:44.467230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16083453-de4e-4956-91b7-80754589224f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:44.472800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6032f6-bcbf-4e4d-99b1-3d88fec0dc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:44.473475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6032f6-bcbf-4e4d-99b1-3d88fec0dc65 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:44.473747 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6032f6-bcbf-4e4d-99b1-3d88fec0dc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:44.474020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6032f6-bcbf-4e4d-99b1-3d88fec0dc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:44.474785 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6032f6-bcbf-4e4d-99b1-3d88fec0dc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:44.476497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6032f6-bcbf-4e4d-99b1-3d88fec0dc65 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:44.487108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1215/2436] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:44.492917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a150f-9d6b-4a0b-b047-63c5cf9e3fe5 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:44.504282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a150f-9d6b-4a0b-b047-63c5cf9e3fe5 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:44.509666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de8f6f-318f-4264-b639-92204ff970a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:44.510436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de8f6f-318f-4264-b639-92204ff970a4 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:44.510767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de8f6f-318f-4264-b639-92204ff970a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:44.511044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de8f6f-318f-4264-b639-92204ff970a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:44.511826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de8f6f-318f-4264-b639-92204ff970a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:44.513440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de8f6f-318f-4264-b639-92204ff970a4 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:44.523843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1222/2437] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:44.528002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6032f6-bcbf-4e4d-99b1-3d88fec0dc65 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:44.537667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6032f6-bcbf-4e4d-99b1-3d88fec0dc65 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:44.542501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af912d5-40d1-44a7-9f11-16c1e63cd6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:44.543628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af912d5-40d1-44a7-9f11-16c1e63cd6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:44.543904 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af912d5-40d1-44a7-9f11-16c1e63cd6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:44.544175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af912d5-40d1-44a7-9f11-16c1e63cd6ad admin admin] PATH_INFO: `/v3/projects/57a340e6e6974e97a19466d361cec053/users/900f14c8a7df4e059ed2f9b643f1c129/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:44.559258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af912d5-40d1-44a7-9f11-16c1e63cd6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a340e6e6974e97a19466d361cec053, user_id=900f14c8a7df4e059ed2f9b643f1c129, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:44.561913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af912d5-40d1-44a7-9f11-16c1e63cd6ad admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:44.657596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1216/2438] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:44 2022] PUT /identity/v3/projects/57a340e6e6974e97a19466d361cec053/users/900f14c8a7df4e059ed2f9b643f1c129/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:44.667795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa17b09-3a80-4ebe-aee6-3c8c11beaeae None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:44.668347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa17b09-3a80-4ebe-aee6-3c8c11beaeae None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:44.668624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa17b09-3a80-4ebe-aee6-3c8c11beaeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:44.697533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-aaa17b09-3a80-4ebe-aee6-3c8c11beaeae None None] MFA Rules not processed for user `900f14c8a7df4e059ed2f9b643f1c129`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:44.698838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-aaa17b09-3a80-4ebe-aee6-3c8c11beaeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:44.792579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1223/2439] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:44.798264 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af912d5-40d1-44a7-9f11-16c1e63cd6ad admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:44.873662 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af912d5-40d1-44a7-9f11-16c1e63cd6ad admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:44.878458 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1780700-d240-48bc-8440-70536999ae91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:44.879129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1780700-d240-48bc-8440-70536999ae91 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:44.879412 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1780700-d240-48bc-8440-70536999ae91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:44.879687 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1780700-d240-48bc-8440-70536999ae91 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:44.880380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1780700-d240-48bc-8440-70536999ae91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:44.882361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1780700-d240-48bc-8440-70536999ae91 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:45.017757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1217/2440] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:44 2022] POST /identity/v3/projects => generated 409 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:45.021810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa17b09-3a80-4ebe-aee6-3c8c11beaeae None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:45.032438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa17b09-3a80-4ebe-aee6-3c8c11beaeae None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:45.107688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf138ae4-ab59-45b8-8225-2ef8283f7ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:45.108394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf138ae4-ab59-45b8-8225-2ef8283f7ded admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:45.108651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf138ae4-ab59-45b8-8225-2ef8283f7ded admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:45.108929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf138ae4-ab59-45b8-8225-2ef8283f7ded admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:45.109628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf138ae4-ab59-45b8-8225-2ef8283f7ded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:45.111616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf138ae4-ab59-45b8-8225-2ef8283f7ded admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:45.215217 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1224/2441] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:45 2022] POST /identity/v3/users => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:45.216562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1780700-d240-48bc-8440-70536999ae91 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:45.228417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1780700-d240-48bc-8440-70536999ae91 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:45.238141 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b780e8f-9d93-4901-8be0-980742fa4ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:45.238919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b780e8f-9d93-4901-8be0-980742fa4ecc admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:45.239352 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b780e8f-9d93-4901-8be0-980742fa4ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:45.239475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b780e8f-9d93-4901-8be0-980742fa4ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:45.240152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b780e8f-9d93-4901-8be0-980742fa4ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:45.241773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b780e8f-9d93-4901-8be0-980742fa4ecc admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:45.254644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1218/2442] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:45.258764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf138ae4-ab59-45b8-8225-2ef8283f7ded admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:45.269104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf138ae4-ab59-45b8-8225-2ef8283f7ded admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:45.274832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879c5fc-3daa-487e-8f94-2bd0cd9fa7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:45.275720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879c5fc-3daa-487e-8f94-2bd0cd9fa7de admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:45.276001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879c5fc-3daa-487e-8f94-2bd0cd9fa7de admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:45.276277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879c5fc-3daa-487e-8f94-2bd0cd9fa7de admin admin] PATH_INFO: `/v3/projects/0a68867a789b410b8587deb3cc66731b/users/bd856780faf94b28ae85d03ec70ec424/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:45.289508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:45.294021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879c5fc-3daa-487e-8f94-2bd0cd9fa7de admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a68867a789b410b8587deb3cc66731b, user_id=bd856780faf94b28ae85d03ec70ec424, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:45.304338 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879c5fc-3daa-487e-8f94-2bd0cd9fa7de admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:45.370415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1225/2443] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:45 2022] PUT /identity/v3/projects/0a68867a789b410b8587deb3cc66731b/users/bd856780faf94b28ae85d03ec70ec424/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:45.374544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b780e8f-9d93-4901-8be0-980742fa4ecc admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:45.384546 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b780e8f-9d93-4901-8be0-980742fa4ecc admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:45.397139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:45.439637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1a1e6-0414-4100-9a5f-a3ce76f17e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:45.440303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1a1e6-0414-4100-9a5f-a3ce76f17e1d admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:45.440587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1a1e6-0414-4100-9a5f-a3ce76f17e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:45.440862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1a1e6-0414-4100-9a5f-a3ce76f17e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:45.441588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1a1e6-0414-4100-9a5f-a3ce76f17e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:45.443243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1a1e6-0414-4100-9a5f-a3ce76f17e1d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:45.452752 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1219/2444] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:45.456979 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879c5fc-3daa-487e-8f94-2bd0cd9fa7de admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:45.467712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879c5fc-3daa-487e-8f94-2bd0cd9fa7de admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:45.478298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b6812-7153-41e6-a807-6ead7c042be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:45.479239 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b6812-7153-41e6-a807-6ead7c042be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:45.479517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b6812-7153-41e6-a807-6ead7c042be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:45.484832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b6812-7153-41e6-a807-6ead7c042be3 admin admin] PATH_INFO: `/v3/projects/0a68867a789b410b8587deb3cc66731b/users/bd856780faf94b28ae85d03ec70ec424/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:45.484832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b6812-7153-41e6-a807-6ead7c042be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a68867a789b410b8587deb3cc66731b, user_id=bd856780faf94b28ae85d03ec70ec424, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:45.484832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b6812-7153-41e6-a807-6ead7c042be3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:45.495121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1226/2445] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:45 2022] PUT /identity/v3/projects/0a68867a789b410b8587deb3cc66731b/users/bd856780faf94b28ae85d03ec70ec424/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:45.507851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be9ab2-a0e6-4217-b7e7-d188cc31410e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:45.507851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be9ab2-a0e6-4217-b7e7-d188cc31410e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:45.507851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be9ab2-a0e6-4217-b7e7-d188cc31410e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:45.537597 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-35be9ab2-a0e6-4217-b7e7-d188cc31410e None None] MFA Rules not processed for user `bd856780faf94b28ae85d03ec70ec424`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:45.540002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-35be9ab2-a0e6-4217-b7e7-d188cc31410e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:45.606864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1220/2446] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:45 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:45.617574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b6812-7153-41e6-a807-6ead7c042be3 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:45.619301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9c0b6812-7153-41e6-a807-6ead7c042be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:45.676347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b6812-7153-41e6-a807-6ead7c042be3 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:45.680175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa1f1e-ca26-4485-9c72-ab3a0aa53303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:45.680839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa1f1e-ca26-4485-9c72-ab3a0aa53303 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:45.681123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa1f1e-ca26-4485-9c72-ab3a0aa53303 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:45.681401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa1f1e-ca26-4485-9c72-ab3a0aa53303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:45.691579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa1f1e-ca26-4485-9c72-ab3a0aa53303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:45.693054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa1f1e-ca26-4485-9c72-ab3a0aa53303 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:45.704756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1227/2447] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:45 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:46.077659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be9ab2-a0e6-4217-b7e7-d188cc31410e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:46.087203 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be9ab2-a0e6-4217-b7e7-d188cc31410e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:46.090812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed0a3d-6cf6-46e4-b39f-4f4061a94728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:46.091449 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed0a3d-6cf6-46e4-b39f-4f4061a94728 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:46.091733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed0a3d-6cf6-46e4-b39f-4f4061a94728 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:46.092010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed0a3d-6cf6-46e4-b39f-4f4061a94728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:46.148140 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed0a3d-6cf6-46e4-b39f-4f4061a94728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:46.149580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed0a3d-6cf6-46e4-b39f-4f4061a94728 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:46.158383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1221/2448] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:46 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:47.198297 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa1f1e-ca26-4485-9c72-ab3a0aa53303 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:47.199253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-00fa1f1e-ca26-4485-9c72-ab3a0aa53303 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:47.265335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa1f1e-ca26-4485-9c72-ab3a0aa53303 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:47.270327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:47.271253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:47.271536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:47.271848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] PATH_INFO: `/v3/users/6a7d79f89cde4565b504056ed8a90637` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:47.273005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a7d79f89cde4565b504056ed8a90637)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:47.275630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:47.496918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a7d79f89cde4565b504056ed8a90637'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:47.506824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a7d79f89cde4565b504056ed8a90637'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:47.507662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a7d79f89cde4565b504056ed8a90637'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:47.580027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a7d79f89cde4565b504056ed8a90637'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:47.580697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a7d79f89cde4565b504056ed8a90637'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:47.595104 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:47.600232 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1228/2449] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:47 2022] DELETE /identity/v3/users/6a7d79f89cde4565b504056ed8a90637 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:47.604778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7988cf0-b346-4d46-9153-339e69b9ff58 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:47.605283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7988cf0-b346-4d46-9153-339e69b9ff58 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:47.605565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7988cf0-b346-4d46-9153-339e69b9ff58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:47.623445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a7988cf0-b346-4d46-9153-339e69b9ff58 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:47.624658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a7988cf0-b346-4d46-9153-339e69b9ff58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:47.665453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:47.692244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1222/2450] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:47.703500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:47.705144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:47.782093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3256706-8212-454a-8b9f-bf5af68e2efe admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:47.786443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb9cae-ee32-4e77-b97a-9a11ec6f7a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:47.787249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb9cae-ee32-4e77-b97a-9a11ec6f7a18 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:47.787611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb9cae-ee32-4e77-b97a-9a11ec6f7a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:47.787958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb9cae-ee32-4e77-b97a-9a11ec6f7a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:47.804273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb9cae-ee32-4e77-b97a-9a11ec6f7a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:47.806607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb9cae-ee32-4e77-b97a-9a11ec6f7a18 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:47.829045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1229/2451] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:48.334098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7988cf0-b346-4d46-9153-339e69b9ff58 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:48.335577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a7988cf0-b346-4d46-9153-339e69b9ff58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:48.349168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7988cf0-b346-4d46-9153-339e69b9ff58 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:48.375684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:48.377189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:48.377808 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:48.378379 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] PATH_INFO: `/v3/projects/47d89a55c3b2492c9aa64eb2205e5a0d` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:48.381280 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d89a55c3b2492c9aa64eb2205e5a0d)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:48.388835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:48.476476 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d89a55c3b2492c9aa64eb2205e5a0d', 'request_id': 'req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:48.483533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d89a55c3b2492c9aa64eb2205e5a0d', 'request_id': 'req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:48.489396 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d89a55c3b2492c9aa64eb2205e5a0d', 'request_id': 'req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:48.604475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] The token cache is being invalidate because project 47d89a55c3b2492c9aa64eb2205e5a0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:48.604750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:48.605655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:48.607062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1223/2452] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:48 2022] DELETE /identity/v3/projects/47d89a55c3b2492c9aa64eb2205e5a0d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:48.611017 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb9cae-ee32-4e77-b97a-9a11ec6f7a18 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:48.622844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb9cae-ee32-4e77-b97a-9a11ec6f7a18 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:48.706798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:48.708158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:48.708639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:48.708984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] PATH_INFO: `/v3/users/14fadf4fdd344cd698d9d0fe5a06d25c` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:48.710272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=14fadf4fdd344cd698d9d0fe5a06d25c)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:48.713021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:48.932606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14fadf4fdd344cd698d9d0fe5a06d25c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:48.939536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14fadf4fdd344cd698d9d0fe5a06d25c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:48.940281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14fadf4fdd344cd698d9d0fe5a06d25c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:49.010429 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14fadf4fdd344cd698d9d0fe5a06d25c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:49.011080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-30d99ecc-5676-46c2-aede-aea28ca3ba2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14fadf4fdd344cd698d9d0fe5a06d25c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:49.029851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1230/2453] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:48 2022] DELETE /identity/v3/users/14fadf4fdd344cd698d9d0fe5a06d25c => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:49.427587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:49.429068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:49.443788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3540b-2efc-49d3-9ca3-4e6e85c4e9c4 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:49.501006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:49.501857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:49.502139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:49.502415 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] PATH_INFO: `/v3/projects/5079a0aea9e34ea2a51d01f0d1c65d91` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:49.503633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5079a0aea9e34ea2a51d01f0d1c65d91)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:49.505533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:49.608748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5079a0aea9e34ea2a51d01f0d1c65d91', 'request_id': 'req-465b3534-5fa0-41af-9f25-3c6f32b4f199'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:49.614849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5079a0aea9e34ea2a51d01f0d1c65d91', 'request_id': 'req-465b3534-5fa0-41af-9f25-3c6f32b4f199'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:49.620627 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5079a0aea9e34ea2a51d01f0d1c65d91', 'request_id': 'req-465b3534-5fa0-41af-9f25-3c6f32b4f199'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:49.622282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:49.701217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] The token cache is being invalidate because project 5079a0aea9e34ea2a51d01f0d1c65d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:49.701516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:49.702062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-465b3534-5fa0-41af-9f25-3c6f32b4f199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:49.703643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1224/2454] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:49 2022] DELETE /identity/v3/projects/5079a0aea9e34ea2a51d01f0d1c65d91 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:49.708603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747244ed-4343-4014-a51f-d6a750340da6 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:49.709180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747244ed-4343-4014-a51f-d6a750340da6 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:49.709180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747244ed-4343-4014-a51f-d6a750340da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:49.727655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-747244ed-4343-4014-a51f-d6a750340da6 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:49.728892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-747244ed-4343-4014-a51f-d6a750340da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:49.746881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:49.792499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1231/2455] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:49.801904 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889fb61-1a70-424e-8258-b29b08a1fe45 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:49.802192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889fb61-1a70-424e-8258-b29b08a1fe45 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:49.802504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889fb61-1a70-424e-8258-b29b08a1fe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:49.821441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-8889fb61-1a70-424e-8258-b29b08a1fe45 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:49.822690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8889fb61-1a70-424e-8258-b29b08a1fe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:49.830539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1225/2456] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:49.834703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747244ed-4343-4014-a51f-d6a750340da6 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:49.844907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747244ed-4343-4014-a51f-d6a750340da6 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:49.849664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad438d-37bb-4323-b8ae-4c864f107237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:49.850571 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad438d-37bb-4323-b8ae-4c864f107237 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:49.850855 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad438d-37bb-4323-b8ae-4c864f107237 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:49.851140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad438d-37bb-4323-b8ae-4c864f107237 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:49.851894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad438d-37bb-4323-b8ae-4c864f107237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:49.852206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad438d-37bb-4323-b8ae-4c864f107237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:49.853850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad438d-37bb-4323-b8ae-4c864f107237 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:49.865961 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1232/2457] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:05:49.873274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889fb61-1a70-424e-8258-b29b08a1fe45 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:49.882861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889fb61-1a70-424e-8258-b29b08a1fe45 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:49.888364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77e0b8-29a5-4054-824e-7d1828d931b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:49.889198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77e0b8-29a5-4054-824e-7d1828d931b5 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:49.889323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77e0b8-29a5-4054-824e-7d1828d931b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:49.889599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77e0b8-29a5-4054-824e-7d1828d931b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:49.890299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77e0b8-29a5-4054-824e-7d1828d931b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:49.892272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77e0b8-29a5-4054-824e-7d1828d931b5 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:49.983220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1226/2458] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:49 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:49.987224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad438d-37bb-4323-b8ae-4c864f107237 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:49.998043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad438d-37bb-4323-b8ae-4c864f107237 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:50.052014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4f6ff-aea2-4749-95ec-5d86fe83701e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:50.052677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4f6ff-aea2-4749-95ec-5d86fe83701e admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.052993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4f6ff-aea2-4749-95ec-5d86fe83701e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.053240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4f6ff-aea2-4749-95ec-5d86fe83701e admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:50.053919 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4f6ff-aea2-4749-95ec-5d86fe83701e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:50.056405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4f6ff-aea2-4749-95ec-5d86fe83701e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:50.139458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1233/2459] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:49 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:50.140778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77e0b8-29a5-4054-824e-7d1828d931b5 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:50.151483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77e0b8-29a5-4054-824e-7d1828d931b5 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:50.156583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bde17-504c-4c59-b2c6-e17a29007b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:50.157248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bde17-504c-4c59-b2c6-e17a29007b34 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.157534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bde17-504c-4c59-b2c6-e17a29007b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.157854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017bde17-504c-4c59-b2c6-e17a29007b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:50.158628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bde17-504c-4c59-b2c6-e17a29007b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:50.160244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017bde17-504c-4c59-b2c6-e17a29007b34 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:50.173622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1227/2460] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:05:50.177835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4f6ff-aea2-4749-95ec-5d86fe83701e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:50.190016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4f6ff-aea2-4749-95ec-5d86fe83701e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:50.194913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fb9ab-d9e4-48ef-a101-bb59b7e4f0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:50.195605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fb9ab-d9e4-48ef-a101-bb59b7e4f0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.195930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fb9ab-d9e4-48ef-a101-bb59b7e4f0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.196211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fb9ab-d9e4-48ef-a101-bb59b7e4f0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:50.197065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fb9ab-d9e4-48ef-a101-bb59b7e4f0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:50.198707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fb9ab-d9e4-48ef-a101-bb59b7e4f0d1 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:50.209675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1234/2461] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:50.214207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bde17-504c-4c59-b2c6-e17a29007b34 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:50.224554 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017bde17-504c-4c59-b2c6-e17a29007b34 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:50.230080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bd661-b3f5-4267-8e31-0601a6c7b9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:50.231120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bd661-b3f5-4267-8e31-0601a6c7b9b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.231407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bd661-b3f5-4267-8e31-0601a6c7b9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.231686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bd661-b3f5-4267-8e31-0601a6c7b9b4 admin admin] PATH_INFO: `/v3/projects/131239796d884d9a9982edcbd70e18dc/users/e7f761cfae6449fd9d94b6c84913c9ee/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:50.248179 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bd661-b3f5-4267-8e31-0601a6c7b9b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=131239796d884d9a9982edcbd70e18dc, user_id=e7f761cfae6449fd9d94b6c84913c9ee, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:50.250094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bd661-b3f5-4267-8e31-0601a6c7b9b4 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:50.313954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1228/2462] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:50 2022] PUT /identity/v3/projects/131239796d884d9a9982edcbd70e18dc/users/e7f761cfae6449fd9d94b6c84913c9ee/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:50.324723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05f8e3-b0aa-4761-9c7d-f0ca399eddeb None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.325051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05f8e3-b0aa-4761-9c7d-f0ca399eddeb None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.325319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05f8e3-b0aa-4761-9c7d-f0ca399eddeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:50.354720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ce05f8e3-b0aa-4761-9c7d-f0ca399eddeb None None] MFA Rules not processed for user `e7f761cfae6449fd9d94b6c84913c9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:50.355978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ce05f8e3-b0aa-4761-9c7d-f0ca399eddeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:50.422243 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1235/2463] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:50.427445 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bd661-b3f5-4267-8e31-0601a6c7b9b4 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:50.498457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bd661-b3f5-4267-8e31-0601a6c7b9b4 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:50.503307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315d02d-c3a4-4da9-86da-53e3f632f575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:50.503974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315d02d-c3a4-4da9-86da-53e3f632f575 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.504261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315d02d-c3a4-4da9-86da-53e3f632f575 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.504540 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315d02d-c3a4-4da9-86da-53e3f632f575 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:50.505233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315d02d-c3a4-4da9-86da-53e3f632f575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:50.507326 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315d02d-c3a4-4da9-86da-53e3f632f575 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:50.603951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1229/2464] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:50 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:50.605157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05f8e3-b0aa-4761-9c7d-f0ca399eddeb None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:50.616008 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05f8e3-b0aa-4761-9c7d-f0ca399eddeb None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:50.628283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:50.674483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082da86a-a372-4130-8171-78ad5b52e7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:50.675191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da86a-a372-4130-8171-78ad5b52e7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.675459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da86a-a372-4130-8171-78ad5b52e7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.675798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da86a-a372-4130-8171-78ad5b52e7b0 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:50.676489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082da86a-a372-4130-8171-78ad5b52e7b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:50.678412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082da86a-a372-4130-8171-78ad5b52e7b0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:50.776183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1236/2465] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:50 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:50.779667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315d02d-c3a4-4da9-86da-53e3f632f575 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:50.790766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315d02d-c3a4-4da9-86da-53e3f632f575 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:50.795923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9d6e3-59f0-4dee-a2f7-0fa9cccea123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:50.796584 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9d6e3-59f0-4dee-a2f7-0fa9cccea123 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.796869 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9d6e3-59f0-4dee-a2f7-0fa9cccea123 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.797301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9d6e3-59f0-4dee-a2f7-0fa9cccea123 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:50.797908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9d6e3-59f0-4dee-a2f7-0fa9cccea123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:50.801981 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9d6e3-59f0-4dee-a2f7-0fa9cccea123 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:50.811902 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1230/2466] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:50.817107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082da86a-a372-4130-8171-78ad5b52e7b0 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:50.826189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082da86a-a372-4130-8171-78ad5b52e7b0 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:50.831197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c92724-6746-4d69-ab00-04ab9b4b17c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:50.831877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92724-6746-4d69-ab00-04ab9b4b17c3 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.832165 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92724-6746-4d69-ab00-04ab9b4b17c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.832847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92724-6746-4d69-ab00-04ab9b4b17c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:50.833216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c92724-6746-4d69-ab00-04ab9b4b17c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:50.834848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c92724-6746-4d69-ab00-04ab9b4b17c3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:50.846047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1237/2467] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:50.849991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9d6e3-59f0-4dee-a2f7-0fa9cccea123 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:50.860147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9d6e3-59f0-4dee-a2f7-0fa9cccea123 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:50.864759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8adbcb-cfa9-4c1c-ad00-3900a182d6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:50.865633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8adbcb-cfa9-4c1c-ad00-3900a182d6e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.865919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8adbcb-cfa9-4c1c-ad00-3900a182d6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.866201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8adbcb-cfa9-4c1c-ad00-3900a182d6e3 admin admin] PATH_INFO: `/v3/projects/78e4252316c04c57999f8cf9c55cf892/users/63a2f4abbdfe43918c164fdad7903c51/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:50.880555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8adbcb-cfa9-4c1c-ad00-3900a182d6e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e4252316c04c57999f8cf9c55cf892, user_id=63a2f4abbdfe43918c164fdad7903c51, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:50.883762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8adbcb-cfa9-4c1c-ad00-3900a182d6e3 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:50.976334 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1231/2468] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:50 2022] PUT /identity/v3/projects/78e4252316c04c57999f8cf9c55cf892/users/63a2f4abbdfe43918c164fdad7903c51/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:50.986829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cf957-582b-4364-8750-98248dd9270e None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:50.987119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cf957-582b-4364-8750-98248dd9270e None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:50.987400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cf957-582b-4364-8750-98248dd9270e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:51.013867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-cb2cf957-582b-4364-8750-98248dd9270e None None] MFA Rules not processed for user `63a2f4abbdfe43918c164fdad7903c51`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:51.015141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-cb2cf957-582b-4364-8750-98248dd9270e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:51.077161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1238/2469] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:51.081617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8adbcb-cfa9-4c1c-ad00-3900a182d6e3 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:51.122951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:51.142967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8adbcb-cfa9-4c1c-ad00-3900a182d6e3 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:51.147641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78066f9c-8944-492d-9dfa-e3f1c4e9344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:51.148294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78066f9c-8944-492d-9dfa-e3f1c4e9344e admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:51.148580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78066f9c-8944-492d-9dfa-e3f1c4e9344e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:51.148862 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78066f9c-8944-492d-9dfa-e3f1c4e9344e admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:51.149559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78066f9c-8944-492d-9dfa-e3f1c4e9344e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:51.151515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78066f9c-8944-492d-9dfa-e3f1c4e9344e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:51.249994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1232/2470] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:51 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:51.254207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cf957-582b-4364-8750-98248dd9270e None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:51.264901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cf957-582b-4364-8750-98248dd9270e None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:51.324657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa86e56-7954-4ff0-8f7b-612032ac756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:51.325363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa86e56-7954-4ff0-8f7b-612032ac756e admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:51.325641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa86e56-7954-4ff0-8f7b-612032ac756e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:51.325914 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa86e56-7954-4ff0-8f7b-612032ac756e admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:51.326656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa86e56-7954-4ff0-8f7b-612032ac756e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:51.328679 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa86e56-7954-4ff0-8f7b-612032ac756e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:51.422312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1239/2471] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:51 2022] POST /identity/v3/users => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:51.426318 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78066f9c-8944-492d-9dfa-e3f1c4e9344e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:51.436839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78066f9c-8944-492d-9dfa-e3f1c4e9344e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:51.441429 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341fdfc-d7cb-4f89-8e25-d86de2c3d20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:51.442164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341fdfc-d7cb-4f89-8e25-d86de2c3d20d admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:51.442448 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341fdfc-d7cb-4f89-8e25-d86de2c3d20d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:51.442753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341fdfc-d7cb-4f89-8e25-d86de2c3d20d admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:51.443478 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341fdfc-d7cb-4f89-8e25-d86de2c3d20d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:51.445092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341fdfc-d7cb-4f89-8e25-d86de2c3d20d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:51.455713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1233/2472] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:51.459714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa86e56-7954-4ff0-8f7b-612032ac756e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:51.470741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa86e56-7954-4ff0-8f7b-612032ac756e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:51.475585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb988ca9-a5a0-4121-afb3-04b8d12f8590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:51.476485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb988ca9-a5a0-4121-afb3-04b8d12f8590 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:51.476762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb988ca9-a5a0-4121-afb3-04b8d12f8590 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:51.477205 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb988ca9-a5a0-4121-afb3-04b8d12f8590 admin admin] PATH_INFO: `/v3/projects/69eec38386e648d8904185a634a50775/users/1b0d5440cbe94aefafe910f08b0a38b6/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:51.489190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:51.493037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb988ca9-a5a0-4121-afb3-04b8d12f8590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69eec38386e648d8904185a634a50775, user_id=1b0d5440cbe94aefafe910f08b0a38b6, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:51.495726 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb988ca9-a5a0-4121-afb3-04b8d12f8590 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:51.588844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1240/2473] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:51 2022] PUT /identity/v3/projects/69eec38386e648d8904185a634a50775/users/1b0d5440cbe94aefafe910f08b0a38b6/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:51.592769 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341fdfc-d7cb-4f89-8e25-d86de2c3d20d admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:51.602930 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341fdfc-d7cb-4f89-8e25-d86de2c3d20d admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:51.659649 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19d25b-94aa-45fb-8990-75746a4dc765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:51.660383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d25b-94aa-45fb-8990-75746a4dc765 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:51.660667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d25b-94aa-45fb-8990-75746a4dc765 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:51.660944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19d25b-94aa-45fb-8990-75746a4dc765 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:51.661828 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19d25b-94aa-45fb-8990-75746a4dc765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:51.663456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19d25b-94aa-45fb-8990-75746a4dc765 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:51.673572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1234/2474] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:51.677758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb988ca9-a5a0-4121-afb3-04b8d12f8590 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:51.688064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb988ca9-a5a0-4121-afb3-04b8d12f8590 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:51.692668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19933e-d362-44f8-81e5-421d6f1fd71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:51.693546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19933e-d362-44f8-81e5-421d6f1fd71e admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:51.693826 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19933e-d362-44f8-81e5-421d6f1fd71e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:51.694176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19933e-d362-44f8-81e5-421d6f1fd71e admin admin] PATH_INFO: `/v3/projects/69eec38386e648d8904185a634a50775/users/1b0d5440cbe94aefafe910f08b0a38b6/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:51.696762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19933e-d362-44f8-81e5-421d6f1fd71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=69eec38386e648d8904185a634a50775, user_id=1b0d5440cbe94aefafe910f08b0a38b6, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:51.699444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19933e-d362-44f8-81e5-421d6f1fd71e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:51.709718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1241/2475] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:51 2022] PUT /identity/v3/projects/69eec38386e648d8904185a634a50775/users/1b0d5440cbe94aefafe910f08b0a38b6/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:51.773842 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a6fee-3127-4ae3-9ea9-a48b502f140e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:51.774861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a6fee-3127-4ae3-9ea9-a48b502f140e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:51.775130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a6fee-3127-4ae3-9ea9-a48b502f140e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:51.800922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-121a6fee-3127-4ae3-9ea9-a48b502f140e None None] MFA Rules not processed for user `1b0d5440cbe94aefafe910f08b0a38b6`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:51.802136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-121a6fee-3127-4ae3-9ea9-a48b502f140e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:51.873512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1235/2476] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:51.887712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19933e-d362-44f8-81e5-421d6f1fd71e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:51.889488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5d19933e-d362-44f8-81e5-421d6f1fd71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:51.959460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19933e-d362-44f8-81e5-421d6f1fd71e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:51.964006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fe93f-946b-43b7-bb61-01494511312d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:51.964758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fe93f-946b-43b7-bb61-01494511312d service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:51.965116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fe93f-946b-43b7-bb61-01494511312d service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:51.965408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fe93f-946b-43b7-bb61-01494511312d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:52.022602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fe93f-946b-43b7-bb61-01494511312d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:52.024083 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fe93f-946b-43b7-bb61-01494511312d service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:52.033283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1242/2477] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:54.011794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a6fee-3127-4ae3-9ea9-a48b502f140e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:54.013252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-121a6fee-3127-4ae3-9ea9-a48b502f140e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:54.025353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a6fee-3127-4ae3-9ea9-a48b502f140e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:54.029003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187db91-9924-4c44-8c0e-954d5f36ab41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:54.031552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187db91-9924-4c44-8c0e-954d5f36ab41 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:54.031552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187db91-9924-4c44-8c0e-954d5f36ab41 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:54.031552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187db91-9924-4c44-8c0e-954d5f36ab41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:54.061241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:54.086972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187db91-9924-4c44-8c0e-954d5f36ab41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:54.088409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187db91-9924-4c44-8c0e-954d5f36ab41 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:54.097166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1236/2478] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:54.814779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fe93f-946b-43b7-bb61-01494511312d service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:54.816089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7f1fe93f-946b-43b7-bb61-01494511312d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:54.873051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fe93f-946b-43b7-bb61-01494511312d service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:54.876639 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af8eb4-6003-4709-a4df-7d75308a70c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:54.877288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af8eb4-6003-4709-a4df-7d75308a70c5 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:54.877602 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af8eb4-6003-4709-a4df-7d75308a70c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:54.877862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af8eb4-6003-4709-a4df-7d75308a70c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:54.879652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-13af8eb4-6003-4709-a4df-7d75308a70c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:54.929452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:54.950732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af8eb4-6003-4709-a4df-7d75308a70c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:54.952155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af8eb4-6003-4709-a4df-7d75308a70c5 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:54.962407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1243/2479] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:55.727338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187db91-9924-4c44-8c0e-954d5f36ab41 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:55.736387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187db91-9924-4c44-8c0e-954d5f36ab41 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:55.740374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30ec00-b1b7-4970-abbb-b814fbe0c220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:55.741021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30ec00-b1b7-4970-abbb-b814fbe0c220 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:55.741303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30ec00-b1b7-4970-abbb-b814fbe0c220 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:55.741582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30ec00-b1b7-4970-abbb-b814fbe0c220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:55.751350 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30ec00-b1b7-4970-abbb-b814fbe0c220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:55.752784 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30ec00-b1b7-4970-abbb-b814fbe0c220 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:55.763757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1237/2480] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:56.420895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af8eb4-6003-4709-a4df-7d75308a70c5 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:56.422532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-13af8eb4-6003-4709-a4df-7d75308a70c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:56.436203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af8eb4-6003-4709-a4df-7d75308a70c5 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:56.441605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:56.442534 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:56.442840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:56.443201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] PATH_INFO: `/v3/users/900f14c8a7df4e059ed2f9b643f1c129` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:56.444437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=900f14c8a7df4e059ed2f9b643f1c129)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:56.446459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:56.629585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '900f14c8a7df4e059ed2f9b643f1c129'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:56.649291 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '900f14c8a7df4e059ed2f9b643f1c129'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:56.650252 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '900f14c8a7df4e059ed2f9b643f1c129'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:56.718778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '900f14c8a7df4e059ed2f9b643f1c129'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:56.719446 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '900f14c8a7df4e059ed2f9b643f1c129'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:56.741875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1244/2481] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:56 2022] DELETE /identity/v3/users/900f14c8a7df4e059ed2f9b643f1c129 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:56.746763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800bcde-3e24-402c-b044-646110dbdcfe None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:56.747296 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800bcde-3e24-402c-b044-646110dbdcfe None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:56.747443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800bcde-3e24-402c-b044-646110dbdcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:56.766909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4800bcde-3e24-402c-b044-646110dbdcfe None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:56.768282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4800bcde-3e24-402c-b044-646110dbdcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:56.840031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1238/2482] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:56.849406 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:56.851740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:56.912176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c828f-14d5-4a9e-b7e8-2800ab88a128 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:56.915970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba47c2-4b56-4b8f-bb1f-ddfc5e127eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:56.916677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba47c2-4b56-4b8f-bb1f-ddfc5e127eac service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:56.916958 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba47c2-4b56-4b8f-bb1f-ddfc5e127eac service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:56.917290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba47c2-4b56-4b8f-bb1f-ddfc5e127eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:56.927066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba47c2-4b56-4b8f-bb1f-ddfc5e127eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:56.928575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba47c2-4b56-4b8f-bb1f-ddfc5e127eac service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:56.939285 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1245/2483] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:57.394655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800bcde-3e24-402c-b044-646110dbdcfe None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:57.398695 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4800bcde-3e24-402c-b044-646110dbdcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:57.409400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800bcde-3e24-402c-b044-646110dbdcfe None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:57.430764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:57.432347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:57.432943 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:57.433430 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] PATH_INFO: `/v3/projects/57a340e6e6974e97a19466d361cec053` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:57.435583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a340e6e6974e97a19466d361cec053)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:57.437933 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:57.572248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a340e6e6974e97a19466d361cec053', 'request_id': 'req-be97c638-5010-40b1-bffc-dac577ea5a64'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:57.579534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a340e6e6974e97a19466d361cec053', 'request_id': 'req-be97c638-5010-40b1-bffc-dac577ea5a64'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:57.586001 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a340e6e6974e97a19466d361cec053', 'request_id': 'req-be97c638-5010-40b1-bffc-dac577ea5a64'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:57.689043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] The token cache is being invalidate because project 57a340e6e6974e97a19466d361cec053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:57.689043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:57.689043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:57.689043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1239/2484] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:57 2022] DELETE /identity/v3/projects/57a340e6e6974e97a19466d361cec053 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:57.692325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba47c2-4b56-4b8f-bb1f-ddfc5e127eac service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:57.702530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba47c2-4b56-4b8f-bb1f-ddfc5e127eac service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:57.764473 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:57.778016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:57.779417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:57.779868 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:57.780439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] PATH_INFO: `/v3/users/bd856780faf94b28ae85d03ec70ec424` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:57.782214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd856780faf94b28ae85d03ec70ec424)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:57.784850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:57.989334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd856780faf94b28ae85d03ec70ec424'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:58.001223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd856780faf94b28ae85d03ec70ec424'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:58.002148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd856780faf94b28ae85d03ec70ec424'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:58.090440 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd856780faf94b28ae85d03ec70ec424'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:58.091166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd856780faf94b28ae85d03ec70ec424'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:58.113595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1246/2485] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:57 2022] DELETE /identity/v3/users/bd856780faf94b28ae85d03ec70ec424 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:58.828768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:58.830280 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:58.846418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97c638-5010-40b1-bffc-dac577ea5a64 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:58.854350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:58.855927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:58.911492 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:58.929295 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:58.930712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:58.930992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:58.931267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] PATH_INFO: `/v3/users/6dc49525e3394668b4f1610c4badc144` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:58.932598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc49525e3394668b4f1610c4badc144)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:58.939165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:58.947942 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177c10e-3d37-4f23-a3cc-14c27fb22ab7 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:58.952876 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:58.953743 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:58.954213 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:58.954644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] PATH_INFO: `/v3/projects/0a68867a789b410b8587deb3cc66731b` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:58.955807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a68867a789b410b8587deb3cc66731b)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:58.957764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.128454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc49525e3394668b4f1610c4badc144'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.130193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a68867a789b410b8587deb3cc66731b', 'request_id': 'req-bb596d07-2ab9-410c-977b-c3f4f0c2a967'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.136118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc49525e3394668b4f1610c4badc144'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.138187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a68867a789b410b8587deb3cc66731b', 'request_id': 'req-bb596d07-2ab9-410c-977b-c3f4f0c2a967'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.144912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a68867a789b410b8587deb3cc66731b', 'request_id': 'req-bb596d07-2ab9-410c-977b-c3f4f0c2a967'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.227883 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc49525e3394668b4f1610c4badc144'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.228508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc49525e3394668b4f1610c4badc144'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.229117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3b8d2f59-3671-4650-add5-900b3c007a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc49525e3394668b4f1610c4badc144'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.246835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1240/2486] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:58 2022] DELETE /identity/v3/users/6dc49525e3394668b4f1610c4badc144 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:59.318229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] The token cache is being invalidate because project 0a68867a789b410b8587deb3cc66731b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:59.318541 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.319168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bb596d07-2ab9-410c-977b-c3f4f0c2a967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.320700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1247/2487] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:58 2022] DELETE /identity/v3/projects/0a68867a789b410b8587deb3cc66731b => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:59.325830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04113a-e0b9-4b96-9914-3d8e0ed51436 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.326119 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04113a-e0b9-4b96-9914-3d8e0ed51436 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.326435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04113a-e0b9-4b96-9914-3d8e0ed51436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.344947 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9e04113a-e0b9-4b96-9914-3d8e0ed51436 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:59.346116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9e04113a-e0b9-4b96-9914-3d8e0ed51436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:59.411796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1241/2488] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:59.420135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c62b2-290d-4840-a069-d7ad23d46ffc None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.420716 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c62b2-290d-4840-a069-d7ad23d46ffc None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.420992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c62b2-290d-4840-a069-d7ad23d46ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.434517 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:59.440753 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ac4c62b2-290d-4840-a069-d7ad23d46ffc None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:59.442029 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ac4c62b2-290d-4840-a069-d7ad23d46ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:59.450278 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1248/2489] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:59.454580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04113a-e0b9-4b96-9914-3d8e0ed51436 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.464875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04113a-e0b9-4b96-9914-3d8e0ed51436 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.469549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb0880-e181-426a-a5ee-3912517c4e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.470158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb0880-e181-426a-a5ee-3912517c4e92 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.470439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb0880-e181-426a-a5ee-3912517c4e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.470750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb0880-e181-426a-a5ee-3912517c4e92 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.471475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb0880-e181-426a-a5ee-3912517c4e92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:59.471781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb0880-e181-426a-a5ee-3912517c4e92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.473414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb0880-e181-426a-a5ee-3912517c4e92 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.485449 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1242/2490] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:05:59.489612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c62b2-290d-4840-a069-d7ad23d46ffc None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.497304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb0880-e181-426a-a5ee-3912517c4e92 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.499290 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-15eb0880-e181-426a-a5ee-3912517c4e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:59.501577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c62b2-290d-4840-a069-d7ad23d46ffc None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.506768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce56aa0-5eaa-48ff-a40f-b27907f9c8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.507617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce56aa0-5eaa-48ff-a40f-b27907f9c8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.507725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce56aa0-5eaa-48ff-a40f-b27907f9c8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.508295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce56aa0-5eaa-48ff-a40f-b27907f9c8e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.509223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce56aa0-5eaa-48ff-a40f-b27907f9c8e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.511253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce56aa0-5eaa-48ff-a40f-b27907f9c8e6 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.514291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb0880-e181-426a-a5ee-3912517c4e92 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.519906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.520959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.521251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.521515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] PATH_INFO: `/v3/projects/488e87a8233e4e69a07cd8efba37cad7` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.522795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] RBAC: Authorizing `identity:delete_project(project_id=488e87a8233e4e69a07cd8efba37cad7)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.528667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.612868 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1249/2491] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:59 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:59.617153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce56aa0-5eaa-48ff-a40f-b27907f9c8e6 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.655776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:59.662832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '488e87a8233e4e69a07cd8efba37cad7', 'request_id': 'req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.672382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '488e87a8233e4e69a07cd8efba37cad7', 'request_id': 'req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.677823 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '488e87a8233e4e69a07cd8efba37cad7', 'request_id': 'req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.687547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce56aa0-5eaa-48ff-a40f-b27907f9c8e6 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.743817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce889ee3-f13f-4f8c-ac79-2fb27c6c9eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.744488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce889ee3-f13f-4f8c-ac79-2fb27c6c9eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.744771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce889ee3-f13f-4f8c-ac79-2fb27c6c9eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.745052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce889ee3-f13f-4f8c-ac79-2fb27c6c9eb5 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.745732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce889ee3-f13f-4f8c-ac79-2fb27c6c9eb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.747679 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce889ee3-f13f-4f8c-ac79-2fb27c6c9eb5 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.786866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] The token cache is being invalidate because project 488e87a8233e4e69a07cd8efba37cad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:59.787153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.787788 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-0fb44e67-d76c-4578-be9a-4dddf6b2c05b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.790041 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1243/2492] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:59 2022] DELETE /identity/v3/projects/488e87a8233e4e69a07cd8efba37cad7 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:59.794939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48938-a3b9-40bb-9d94-c9559bb60dc4 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.795225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48938-a3b9-40bb-9d94-c9559bb60dc4 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.795505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48938-a3b9-40bb-9d94-c9559bb60dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.812975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-39c48938-a3b9-40bb-9d94-c9559bb60dc4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:59.814339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-39c48938-a3b9-40bb-9d94-c9559bb60dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:59.822816 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1244/2493] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:59.830551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c3a12-b900-47ef-ad66-c2121ea841f1 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.831113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c3a12-b900-47ef-ad66-c2121ea841f1 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.831113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c3a12-b900-47ef-ad66-c2121ea841f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.848904 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-502c3a12-b900-47ef-ad66-c2121ea841f1 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:59.850139 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-502c3a12-b900-47ef-ad66-c2121ea841f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:59.855038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1250/2494] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:59 2022] POST /identity/v3/users => generated 294 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:59.858305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1245/2495] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:59.862101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c3a12-b900-47ef-ad66-c2121ea841f1 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.862251 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce889ee3-f13f-4f8c-ac79-2fb27c6c9eb5 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.863873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ce889ee3-f13f-4f8c-ac79-2fb27c6c9eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:59.871255 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c3a12-b900-47ef-ad66-c2121ea841f1 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.875971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbb428-a8fe-40bf-89de-b1aca87897ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.876140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce889ee3-f13f-4f8c-ac79-2fb27c6c9eb5 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.876613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb428-a8fe-40bf-89de-b1aca87897ca admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.876901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb428-a8fe-40bf-89de-b1aca87897ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.877181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb428-a8fe-40bf-89de-b1aca87897ca admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.877907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbb428-a8fe-40bf-89de-b1aca87897ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:59.878263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbb428-a8fe-40bf-89de-b1aca87897ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.879948 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbb428-a8fe-40bf-89de-b1aca87897ca admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.881165 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fe657-7bfe-4e7f-8e65-00e3e6bd756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.881827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fe657-7bfe-4e7f-8e65-00e3e6bd756e admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.882143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fe657-7bfe-4e7f-8e65-00e3e6bd756e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.882421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fe657-7bfe-4e7f-8e65-00e3e6bd756e admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.883698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fe657-7bfe-4e7f-8e65-00e3e6bd756e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.885405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fe657-7bfe-4e7f-8e65-00e3e6bd756e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.898182 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1246/2496] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:05:59.898857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1251/2497] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:59.904607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbb428-a8fe-40bf-89de-b1aca87897ca admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.904803 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fe657-7bfe-4e7f-8e65-00e3e6bd756e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.917662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fe657-7bfe-4e7f-8e65-00e3e6bd756e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.920676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbb428-a8fe-40bf-89de-b1aca87897ca admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.923260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb77e74-0af4-4873-b2af-a5f766363c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.923993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb77e74-0af4-4873-b2af-a5f766363c29 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.924223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb77e74-0af4-4873-b2af-a5f766363c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.924507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb77e74-0af4-4873-b2af-a5f766363c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.925233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb77e74-0af4-4873-b2af-a5f766363c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.925964 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa250c5-9f66-4640-89ec-a7ae714eb11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.926682 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa250c5-9f66-4640-89ec-a7ae714eb11f admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.926879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb77e74-0af4-4873-b2af-a5f766363c29 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.927031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa250c5-9f66-4640-89ec-a7ae714eb11f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.927306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa250c5-9f66-4640-89ec-a7ae714eb11f admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.928025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa250c5-9f66-4640-89ec-a7ae714eb11f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.929926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa250c5-9f66-4640-89ec-a7ae714eb11f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.937887 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1252/2498] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:59.942069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb77e74-0af4-4873-b2af-a5f766363c29 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.952487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb77e74-0af4-4873-b2af-a5f766363c29 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.957220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa5552-822f-4b9b-8206-e450b7fef664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.958086 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa5552-822f-4b9b-8206-e450b7fef664 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.958367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa5552-822f-4b9b-8206-e450b7fef664 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.958673 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa5552-822f-4b9b-8206-e450b7fef664 admin admin] PATH_INFO: `/v3/projects/1fbc13137015443f8ad1a678bceed57d/users/05df878a28f9490ca24e9692589ee631/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.974019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa5552-822f-4b9b-8206-e450b7fef664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fbc13137015443f8ad1a678bceed57d, user_id=05df878a28f9490ca24e9692589ee631, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.977279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa5552-822f-4b9b-8206-e450b7fef664 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.028269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1247/2499] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:59 2022] POST /identity/v3/projects => generated 405 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:00.032430 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa250c5-9f66-4640-89ec-a7ae714eb11f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.088719 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1253/2500] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:59 2022] PUT /identity/v3/projects/1fbc13137015443f8ad1a678bceed57d/users/05df878a28f9490ca24e9692589ee631/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:00.097796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa250c5-9f66-4640-89ec-a7ae714eb11f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.099501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678502cf-be18-4877-87c1-57b9f2322b66 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.099734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678502cf-be18-4877-87c1-57b9f2322b66 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.100018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678502cf-be18-4877-87c1-57b9f2322b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.130162 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-678502cf-be18-4877-87c1-57b9f2322b66 None None] MFA Rules not processed for user `05df878a28f9490ca24e9692589ee631`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:00.131532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-678502cf-be18-4877-87c1-57b9f2322b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:00.165302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebf119-b480-4a39-b16e-c87c534d8e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.166251 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebf119-b480-4a39-b16e-c87c534d8e40 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.166569 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebf119-b480-4a39-b16e-c87c534d8e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.166868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebf119-b480-4a39-b16e-c87c534d8e40 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.167560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebf119-b480-4a39-b16e-c87c534d8e40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.169769 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebf119-b480-4a39-b16e-c87c534d8e40 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.197046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:00.206412 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1254/2501] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:00.211019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678502cf-be18-4877-87c1-57b9f2322b66 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.222638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678502cf-be18-4877-87c1-57b9f2322b66 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.227444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31c9c6-c737-48a5-b4be-5be429c9d674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.231043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31c9c6-c737-48a5-b4be-5be429c9d674 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.231043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31c9c6-c737-48a5-b4be-5be429c9d674 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.231043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31c9c6-c737-48a5-b4be-5be429c9d674 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.231043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31c9c6-c737-48a5-b4be-5be429c9d674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.231643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31c9c6-c737-48a5-b4be-5be429c9d674 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.285181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1248/2502] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/users => generated 303 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:00.289526 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebf119-b480-4a39-b16e-c87c534d8e40 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.301007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebf119-b480-4a39-b16e-c87c534d8e40 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.306204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06397c-b1a3-4cf2-8ad9-a94c6be2d19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.306900 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06397c-b1a3-4cf2-8ad9-a94c6be2d19e admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.307181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06397c-b1a3-4cf2-8ad9-a94c6be2d19e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.307456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06397c-b1a3-4cf2-8ad9-a94c6be2d19e admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.308184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06397c-b1a3-4cf2-8ad9-a94c6be2d19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.309789 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06397c-b1a3-4cf2-8ad9-a94c6be2d19e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.322115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1249/2503] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:00.326269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06397c-b1a3-4cf2-8ad9-a94c6be2d19e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.335474 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1255/2504] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:00.338024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06397c-b1a3-4cf2-8ad9-a94c6be2d19e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.339109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31c9c6-c737-48a5-b4be-5be429c9d674 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.399121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00405ae7-41b6-4681-bb69-2dbaf8e859c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.399726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00405ae7-41b6-4681-bb69-2dbaf8e859c8 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.400012 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00405ae7-41b6-4681-bb69-2dbaf8e859c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.400298 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00405ae7-41b6-4681-bb69-2dbaf8e859c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.401030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00405ae7-41b6-4681-bb69-2dbaf8e859c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.402677 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00405ae7-41b6-4681-bb69-2dbaf8e859c8 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.403599 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31c9c6-c737-48a5-b4be-5be429c9d674 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.408402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8d8ea-c361-46a1-b3d2-e9127c4b36e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.409086 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8d8ea-c361-46a1-b3d2-e9127c4b36e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.409374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8d8ea-c361-46a1-b3d2-e9127c4b36e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.409655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8d8ea-c361-46a1-b3d2-e9127c4b36e6 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.410518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8d8ea-c361-46a1-b3d2-e9127c4b36e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.412463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8d8ea-c361-46a1-b3d2-e9127c4b36e6 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.420125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1250/2505] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:00.421178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00405ae7-41b6-4681-bb69-2dbaf8e859c8 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.435172 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00405ae7-41b6-4681-bb69-2dbaf8e859c8 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.440205 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e4077-3890-40f5-bc1c-95296257a146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.441107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4077-3890-40f5-bc1c-95296257a146 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.441393 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4077-3890-40f5-bc1c-95296257a146 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.441672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4077-3890-40f5-bc1c-95296257a146 admin admin] PATH_INFO: `/v3/projects/3f4846e5ebaf44b781f9a3b54ed02c7c/users/6e6fd14d39354130a5931697f15013c9/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.465512 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e4077-3890-40f5-bc1c-95296257a146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f4846e5ebaf44b781f9a3b54ed02c7c, user_id=6e6fd14d39354130a5931697f15013c9, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.468338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e4077-3890-40f5-bc1c-95296257a146 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.509676 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1256/2506] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:00.514912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8d8ea-c361-46a1-b3d2-e9127c4b36e6 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.516721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f5e8d8ea-c361-46a1-b3d2-e9127c4b36e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:00.529262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8d8ea-c361-46a1-b3d2-e9127c4b36e6 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.534171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.535075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.539760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.539760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] PATH_INFO: `/v3/users/e7f761cfae6449fd9d94b6c84913c9ee` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.539760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f761cfae6449fd9d94b6c84913c9ee)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.540177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.551409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1251/2507] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:00 2022] PUT /identity/v3/projects/3f4846e5ebaf44b781f9a3b54ed02c7c/users/6e6fd14d39354130a5931697f15013c9/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:00.553337 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e4077-3890-40f5-bc1c-95296257a146 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.592271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:00.627357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e4077-3890-40f5-bc1c-95296257a146 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.634034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29981bc2-944f-42e9-bb89-5d03da3f2e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.634728 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29981bc2-944f-42e9-bb89-5d03da3f2e45 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.635006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29981bc2-944f-42e9-bb89-5d03da3f2e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.635276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29981bc2-944f-42e9-bb89-5d03da3f2e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.636031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29981bc2-944f-42e9-bb89-5d03da3f2e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.637713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29981bc2-944f-42e9-bb89-5d03da3f2e45 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.648036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1252/2508] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:00.649730 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee80f36-20d8-42dc-833b-1074cd4641d5 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.649958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee80f36-20d8-42dc-833b-1074cd4641d5 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.650233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee80f36-20d8-42dc-833b-1074cd4641d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.677370 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-6ee80f36-20d8-42dc-833b-1074cd4641d5 None None] MFA Rules not processed for user `6e6fd14d39354130a5931697f15013c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:00.678615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-6ee80f36-20d8-42dc-833b-1074cd4641d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:00.744426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1253/2509] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:00.748111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f761cfae6449fd9d94b6c84913c9ee'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:00.748243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee80f36-20d8-42dc-833b-1074cd4641d5 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.753262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f761cfae6449fd9d94b6c84913c9ee'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:00.753978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f761cfae6449fd9d94b6c84913c9ee'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:00.757315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee80f36-20d8-42dc-833b-1074cd4641d5 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.762088 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a90c72-2d33-4b4a-9448-e03168055f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.763211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a90c72-2d33-4b4a-9448-e03168055f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.763493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a90c72-2d33-4b4a-9448-e03168055f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.763772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a90c72-2d33-4b4a-9448-e03168055f35 admin admin] PATH_INFO: `/v3/projects/2c096761bee84569ac2ac78d908fd752/users/413eb745feae43d785f6cfb26d2956d9/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.779751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a90c72-2d33-4b4a-9448-e03168055f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c096761bee84569ac2ac78d908fd752, user_id=413eb745feae43d785f6cfb26d2956d9, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.782429 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a90c72-2d33-4b4a-9448-e03168055f35 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.878762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f761cfae6449fd9d94b6c84913c9ee'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:00.879458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f761cfae6449fd9d94b6c84913c9ee'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:00.885193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1254/2510] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:00 2022] PUT /identity/v3/projects/2c096761bee84569ac2ac78d908fd752/users/413eb745feae43d785f6cfb26d2956d9/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:00.888756 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a90c72-2d33-4b4a-9448-e03168055f35 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.890356 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-33a90c72-2d33-4b4a-9448-e03168055f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:00.895882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:00.900970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1257/2511] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:00 2022] DELETE /identity/v3/users/e7f761cfae6449fd9d94b6c84913c9ee => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:00.904016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.904016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:00.961904 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a90c72-2d33-4b4a-9448-e03168055f35 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.966812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12987fae-4b89-4796-8deb-5f115c775d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.968667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12987fae-4b89-4796-8deb-5f115c775d63 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.968667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12987fae-4b89-4796-8deb-5f115c775d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.968667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12987fae-4b89-4796-8deb-5f115c775d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.968947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef321dde-07c0-4c75-a2ca-b51c32a1d9fe admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.969043 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12987fae-4b89-4796-8deb-5f115c775d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.970741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12987fae-4b89-4796-8deb-5f115c775d63 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.973598 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbffef-1ef7-4d8d-bbd8-1c671131377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.974425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbffef-1ef7-4d8d-bbd8-1c671131377b admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.974748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbffef-1ef7-4d8d-bbd8-1c671131377b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.975028 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbffef-1ef7-4d8d-bbd8-1c671131377b admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.975877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbffef-1ef7-4d8d-bbd8-1c671131377b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.977511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbffef-1ef7-4d8d-bbd8-1c671131377b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.987485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1258/2512] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:00.988823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560281c6-e69b-4149-bf14-60855d02909f None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.989115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560281c6-e69b-4149-bf14-60855d02909f None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.989400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560281c6-e69b-4149-bf14-60855d02909f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.990909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:01.006580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-560281c6-e69b-4149-bf14-60855d02909f None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:01.007785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-560281c6-e69b-4149-bf14-60855d02909f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:01.016039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1259/2513] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:01.016846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560281c6-e69b-4149-bf14-60855d02909f None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.026715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560281c6-e69b-4149-bf14-60855d02909f None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.031676 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414b26f-58d7-4752-8b05-1e7be2129546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.032563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414b26f-58d7-4752-8b05-1e7be2129546 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.032843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414b26f-58d7-4752-8b05-1e7be2129546 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.033116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414b26f-58d7-4752-8b05-1e7be2129546 admin admin] PATH_INFO: `/v3/projects/2c096761bee84569ac2ac78d908fd752/users/413eb745feae43d785f6cfb26d2956d9/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.035148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414b26f-58d7-4752-8b05-1e7be2129546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c096761bee84569ac2ac78d908fd752, user_id=413eb745feae43d785f6cfb26d2956d9, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.037823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414b26f-58d7-4752-8b05-1e7be2129546 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.048362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1260/2514] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:01 2022] PUT /identity/v3/projects/2c096761bee84569ac2ac78d908fd752/users/413eb745feae43d785f6cfb26d2956d9/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:01.049130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414b26f-58d7-4752-8b05-1e7be2129546 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.050713 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0414b26f-58d7-4752-8b05-1e7be2129546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:01.064114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1255/2515] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/projects => generated 405 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:01.065654 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103099f-5bec-438c-8773-7dd27f87c485 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.065941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103099f-5bec-438c-8773-7dd27f87c485 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.066271 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103099f-5bec-438c-8773-7dd27f87c485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.091939 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-8103099f-5bec-438c-8773-7dd27f87c485 None None] MFA Rules not processed for user `413eb745feae43d785f6cfb26d2956d9`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:01.093147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8103099f-5bec-438c-8773-7dd27f87c485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:01.110736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414b26f-58d7-4752-8b05-1e7be2129546 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.159197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28a7ad-53a9-4950-9db7-0d443df14545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.159770 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28a7ad-53a9-4950-9db7-0d443df14545 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.160050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1256/2516] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:01.160175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28a7ad-53a9-4950-9db7-0d443df14545 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.160328 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28a7ad-53a9-4950-9db7-0d443df14545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.163520 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103099f-5bec-438c-8773-7dd27f87c485 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.203582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:01.216096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:01.222360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28a7ad-53a9-4950-9db7-0d443df14545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.223843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28a7ad-53a9-4950-9db7-0d443df14545 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.226666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103099f-5bec-438c-8773-7dd27f87c485 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.232009 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0c483-7c34-4b37-b7e2-d709bacbe680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.232781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0c483-7c34-4b37-b7e2-d709bacbe680 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.233068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0c483-7c34-4b37-b7e2-d709bacbe680 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.233345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0c483-7c34-4b37-b7e2-d709bacbe680 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.234035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0c483-7c34-4b37-b7e2-d709bacbe680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.234186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1261/2517] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:01.235053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28a7ad-53a9-4950-9db7-0d443df14545 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.236247 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0c483-7c34-4b37-b7e2-d709bacbe680 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.236367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0f28a7ad-53a9-4950-9db7-0d443df14545 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:01.295075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28a7ad-53a9-4950-9db7-0d443df14545 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.298784 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1def336-3c14-49ea-b2b4-1ebd14942f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.299479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1def336-3c14-49ea-b2b4-1ebd14942f64 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.299766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1def336-3c14-49ea-b2b4-1ebd14942f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.300040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1def336-3c14-49ea-b2b4-1ebd14942f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.310884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1def336-3c14-49ea-b2b4-1ebd14942f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.313543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1def336-3c14-49ea-b2b4-1ebd14942f64 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.318850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1257/2518] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:01 2022] POST /identity/v3/users => generated 309 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:01.320070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0c483-7c34-4b37-b7e2-d709bacbe680 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.329943 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1262/2519] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:01.332451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0c483-7c34-4b37-b7e2-d709bacbe680 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.338622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd568d-6586-4730-ae75-be5f006e8009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.339409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd568d-6586-4730-ae75-be5f006e8009 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.339798 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd568d-6586-4730-ae75-be5f006e8009 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.340216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd568d-6586-4730-ae75-be5f006e8009 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.341037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd568d-6586-4730-ae75-be5f006e8009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.342750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd568d-6586-4730-ae75-be5f006e8009 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.352798 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1258/2520] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:01.357088 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1def336-3c14-49ea-b2b4-1ebd14942f64 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.367335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1def336-3c14-49ea-b2b4-1ebd14942f64 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.372039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb72e7f-ba9f-406a-8ed7-d8a4f6872ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.372917 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb72e7f-ba9f-406a-8ed7-d8a4f6872ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.373193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb72e7f-ba9f-406a-8ed7-d8a4f6872ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.373464 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb72e7f-ba9f-406a-8ed7-d8a4f6872ee8 admin admin] PATH_INFO: `/v3/projects/b7cc96c0f75c43e09cab01b33520d441/users/5dde614467d0457798e9b22f2ec33fc5/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.388547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb72e7f-ba9f-406a-8ed7-d8a4f6872ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7cc96c0f75c43e09cab01b33520d441, user_id=5dde614467d0457798e9b22f2ec33fc5, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.391238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb72e7f-ba9f-406a-8ed7-d8a4f6872ee8 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.461123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1263/2521] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:01 2022] PUT /identity/v3/projects/b7cc96c0f75c43e09cab01b33520d441/users/5dde614467d0457798e9b22f2ec33fc5/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:01.464069 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd568d-6586-4730-ae75-be5f006e8009 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.476188 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd568d-6586-4730-ae75-be5f006e8009 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.539640 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e363b-aa57-4f8e-90fb-df0f91e325ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.540758 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e363b-aa57-4f8e-90fb-df0f91e325ff admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.541077 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e363b-aa57-4f8e-90fb-df0f91e325ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.541303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e363b-aa57-4f8e-90fb-df0f91e325ff admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.542035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e363b-aa57-4f8e-90fb-df0f91e325ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.543954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e363b-aa57-4f8e-90fb-df0f91e325ff admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.554243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1259/2522] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:01.558266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb72e7f-ba9f-406a-8ed7-d8a4f6872ee8 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.568916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb72e7f-ba9f-406a-8ed7-d8a4f6872ee8 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.574341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42dba3-b1a4-45d2-9421-3d5ece2dcf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.575274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42dba3-b1a4-45d2-9421-3d5ece2dcf05 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.581245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42dba3-b1a4-45d2-9421-3d5ece2dcf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.581245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42dba3-b1a4-45d2-9421-3d5ece2dcf05 admin admin] PATH_INFO: `/v3/projects/b7cc96c0f75c43e09cab01b33520d441/users/5dde614467d0457798e9b22f2ec33fc5/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.581245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42dba3-b1a4-45d2-9421-3d5ece2dcf05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7cc96c0f75c43e09cab01b33520d441, user_id=5dde614467d0457798e9b22f2ec33fc5, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.583198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42dba3-b1a4-45d2-9421-3d5ece2dcf05 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.595781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1264/2523] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:01 2022] PUT /identity/v3/projects/b7cc96c0f75c43e09cab01b33520d441/users/5dde614467d0457798e9b22f2ec33fc5/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:01.605514 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a3037-bd50-40fe-89c7-5caa45bac6e3 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.605864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a3037-bd50-40fe-89c7-5caa45bac6e3 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.606065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a3037-bd50-40fe-89c7-5caa45bac6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.636604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-060a3037-bd50-40fe-89c7-5caa45bac6e3 None None] MFA Rules not processed for user `5dde614467d0457798e9b22f2ec33fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:01.637839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-060a3037-bd50-40fe-89c7-5caa45bac6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:01.671524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42dba3-b1a4-45d2-9421-3d5ece2dcf05 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.673480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-df42dba3-b1a4-45d2-9421-3d5ece2dcf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:01.720463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1260/2524] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:01 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:01.732365 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a3037-bd50-40fe-89c7-5caa45bac6e3 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.751620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42dba3-b1a4-45d2-9421-3d5ece2dcf05 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.761776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.761776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.761776 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.762110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] PATH_INFO: `/v3/projects/131239796d884d9a9982edcbd70e18dc` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.763350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] RBAC: Authorizing `identity:delete_project(project_id=131239796d884d9a9982edcbd70e18dc)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.765348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.796343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a3037-bd50-40fe-89c7-5caa45bac6e3 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.800158 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142443f6-0ac2-48f4-9bd2-0f2a087410cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.800806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142443f6-0ac2-48f4-9bd2-0f2a087410cc service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.801131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142443f6-0ac2-48f4-9bd2-0f2a087410cc service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.801410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142443f6-0ac2-48f4-9bd2-0f2a087410cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.809042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:01.812682 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142443f6-0ac2-48f4-9bd2-0f2a087410cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.812849 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142443f6-0ac2-48f4-9bd2-0f2a087410cc service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.822784 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1261/2525] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:01 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:01.911065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:01.913730 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '131239796d884d9a9982edcbd70e18dc', 'request_id': 'req-e9d17a03-24b7-482d-9c49-0eb19205b88f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:01.924529 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '131239796d884d9a9982edcbd70e18dc', 'request_id': 'req-e9d17a03-24b7-482d-9c49-0eb19205b88f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:01.929698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '131239796d884d9a9982edcbd70e18dc', 'request_id': 'req-e9d17a03-24b7-482d-9c49-0eb19205b88f'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:01.964091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142443f6-0ac2-48f4-9bd2-0f2a087410cc service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.974454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142443f6-0ac2-48f4-9bd2-0f2a087410cc service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:02.015643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] The token cache is being invalidate because project 131239796d884d9a9982edcbd70e18dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:02.015938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.016594 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.018287 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1265/2526] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:01 2022] DELETE /identity/v3/projects/131239796d884d9a9982edcbd70e18dc => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:02.022400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:02.023798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:02.029710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:02.032374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:02.032374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:02.032374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] PATH_INFO: `/v3/users/6e6fd14d39354130a5931697f15013c9` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:02.032661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6fd14d39354130a5931697f15013c9)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:02.034559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:02.088164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d17a03-24b7-482d-9c49-0eb19205b88f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:02.093371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:02.094307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:02.094984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:02.095519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] PATH_INFO: `/v3/users/63a2f4abbdfe43918c164fdad7903c51` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:02.097164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a2f4abbdfe43918c164fdad7903c51)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:02.099181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:02.241799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6fd14d39354130a5931697f15013c9'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.249026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6fd14d39354130a5931697f15013c9'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.308229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a2f4abbdfe43918c164fdad7903c51'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.315194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a2f4abbdfe43918c164fdad7903c51'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.315871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a2f4abbdfe43918c164fdad7903c51'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.317239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6fd14d39354130a5931697f15013c9'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.317861 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6fd14d39354130a5931697f15013c9'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.318465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e232627d-b198-455e-bf03-0c7e13f2294a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6fd14d39354130a5931697f15013c9'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.336923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1262/2527] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:01 2022] DELETE /identity/v3/users/6e6fd14d39354130a5931697f15013c9 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:02.341175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a73e-9c4e-4439-a9c2-5bcdfefb7475 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:02.341461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a73e-9c4e-4439-a9c2-5bcdfefb7475 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:02.341733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a73e-9c4e-4439-a9c2-5bcdfefb7475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:02.358274 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1f97a73e-9c4e-4439-a9c2-5bcdfefb7475 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:02.359509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1f97a73e-9c4e-4439-a9c2-5bcdfefb7475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:02.416462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a2f4abbdfe43918c164fdad7903c51'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.417093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a2f4abbdfe43918c164fdad7903c51'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.418041 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1263/2528] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:02.428416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a73e-9c4e-4439-a9c2-5bcdfefb7475 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:02.430762 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1f97a73e-9c4e-4439-a9c2-5bcdfefb7475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:02.443345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1266/2529] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:02 2022] DELETE /identity/v3/users/63a2f4abbdfe43918c164fdad7903c51 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:02.455772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:02.500654 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a73e-9c4e-4439-a9c2-5bcdfefb7475 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:02.548544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b84b4-075c-47b4-b654-4856111ed8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:02.549193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b84b4-075c-47b4-b654-4856111ed8af service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:02.549475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b84b4-075c-47b4-b654-4856111ed8af service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:02.549747 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b84b4-075c-47b4-b654-4856111ed8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:02.551567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b35b84b4-075c-47b4-b654-4856111ed8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:02.595651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:02.597248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:02.619005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b84b4-075c-47b4-b654-4856111ed8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:02.620621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b84b4-075c-47b4-b654-4856111ed8af service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:02.631347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1264/2530] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:02.678437 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:02.697778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee475cf-9a47-4fcc-8f6e-592b69575c50 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:02.702711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:02.703601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:02.706374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:02.706374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] PATH_INFO: `/v3/users/05df878a28f9490ca24e9692589ee631` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:02.706374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05df878a28f9490ca24e9692589ee631)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:02.707566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:02.947837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b84b4-075c-47b4-b654-4856111ed8af service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:02.949560 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-b35b84b4-075c-47b4-b654-4856111ed8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:02.950772 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05df878a28f9490ca24e9692589ee631'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.957950 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05df878a28f9490ca24e9692589ee631'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.959276 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05df878a28f9490ca24e9692589ee631'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.963841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b84b4-075c-47b4-b654-4856111ed8af service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:02.972074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:02.973184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:02.973407 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:02.973805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] PATH_INFO: `/v3/projects/78e4252316c04c57999f8cf9c55cf892` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:02.975617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e4252316c04c57999f8cf9c55cf892)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:02.977761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.044907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05df878a28f9490ca24e9692589ee631'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.045740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-60e6a3c9-0b5e-4067-8336-cfa56fb790c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05df878a28f9490ca24e9692589ee631'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.063547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1267/2531] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:02 2022] DELETE /identity/v3/users/05df878a28f9490ca24e9692589ee631 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:03.067882 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f701ed-ca1d-4c07-b3c1-eff38e627c03 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.068176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f701ed-ca1d-4c07-b3c1-eff38e627c03 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.068460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f701ed-ca1d-4c07-b3c1-eff38e627c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.085928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c0f701ed-ca1d-4c07-b3c1-eff38e627c03 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:03.088044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c0f701ed-ca1d-4c07-b3c1-eff38e627c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:03.096940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e4252316c04c57999f8cf9c55cf892', 'request_id': 'req-799b228f-3b87-483e-9afe-f981bc8de0d9'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.103248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e4252316c04c57999f8cf9c55cf892', 'request_id': 'req-799b228f-3b87-483e-9afe-f981bc8de0d9'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.108952 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e4252316c04c57999f8cf9c55cf892', 'request_id': 'req-799b228f-3b87-483e-9afe-f981bc8de0d9'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.145267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1268/2532] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:03.155608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f701ed-ca1d-4c07-b3c1-eff38e627c03 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.156969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c0f701ed-ca1d-4c07-b3c1-eff38e627c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:03.217884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f701ed-ca1d-4c07-b3c1-eff38e627c03 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.222143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] The token cache is being invalidate because project 78e4252316c04c57999f8cf9c55cf892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:03.222534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.223164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.225020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1265/2533] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:02 2022] DELETE /identity/v3/projects/78e4252316c04c57999f8cf9c55cf892 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:03.225757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.227006 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:03.228504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef0ebc-915f-4691-85ce-50cf9cab402a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.229281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef0ebc-915f-4691-85ce-50cf9cab402a service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.229567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef0ebc-915f-4691-85ce-50cf9cab402a service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.229914 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef0ebc-915f-4691-85ce-50cf9cab402a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.231849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-81ef0ebc-915f-4691-85ce-50cf9cab402a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:03.237088 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:03.241021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b228f-3b87-483e-9afe-f981bc8de0d9 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.244510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef0ebc-915f-4691-85ce-50cf9cab402a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.245947 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef0ebc-915f-4691-85ce-50cf9cab402a service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.246848 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.247712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.247999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.248279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] PATH_INFO: `/v3/projects/3f4846e5ebaf44b781f9a3b54ed02c7c` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.249561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f4846e5ebaf44b781f9a3b54ed02c7c)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.251521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.255819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1269/2534] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:03.256450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef0ebc-915f-4691-85ce-50cf9cab402a service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.257814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-81ef0ebc-915f-4691-85ce-50cf9cab402a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:03.275354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef0ebc-915f-4691-85ce-50cf9cab402a service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.282835 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.283749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.284035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.287274 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] PATH_INFO: `/v3/users/1b0d5440cbe94aefafe910f08b0a38b6` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.291394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b0d5440cbe94aefafe910f08b0a38b6)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.295090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.314742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:03.557743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f4846e5ebaf44b781f9a3b54ed02c7c', 'request_id': 'req-2b1d395e-5f30-43d5-9445-e6e455c6369f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.558343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b0d5440cbe94aefafe910f08b0a38b6'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.565477 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f4846e5ebaf44b781f9a3b54ed02c7c', 'request_id': 'req-2b1d395e-5f30-43d5-9445-e6e455c6369f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.566268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b0d5440cbe94aefafe910f08b0a38b6'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.567111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b0d5440cbe94aefafe910f08b0a38b6'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.572870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f4846e5ebaf44b781f9a3b54ed02c7c', 'request_id': 'req-2b1d395e-5f30-43d5-9445-e6e455c6369f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.655928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b0d5440cbe94aefafe910f08b0a38b6'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.656542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b0d5440cbe94aefafe910f08b0a38b6'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.676070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1270/2535] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:03 2022] DELETE /identity/v3/users/1b0d5440cbe94aefafe910f08b0a38b6 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:03.701269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.702693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:03.773700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050f6ab-f318-4656-9cf9-07f5406360a5 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.778972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.779843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.780129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.780408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] PATH_INFO: `/v3/projects/1fbc13137015443f8ad1a678bceed57d` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.781616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fbc13137015443f8ad1a678bceed57d)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.783564 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.797688 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] The token cache is being invalidate because project 3f4846e5ebaf44b781f9a3b54ed02c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:03.797871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.798534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.800441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1266/2536] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:03 2022] DELETE /identity/v3/projects/3f4846e5ebaf44b781f9a3b54ed02c7c => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:03.804174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.805566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:03.867033 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d395e-5f30-43d5-9445-e6e455c6369f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.871597 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.872457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.872732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.873003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] PATH_INFO: `/v3/users/5dde614467d0457798e9b22f2ec33fc5` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.874173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dde614467d0457798e9b22f2ec33fc5)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.876101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.963997 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fbc13137015443f8ad1a678bceed57d', 'request_id': 'req-6659ae26-9796-4439-8715-883ef3f24901'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.971706 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fbc13137015443f8ad1a678bceed57d', 'request_id': 'req-6659ae26-9796-4439-8715-883ef3f24901'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:03.978077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fbc13137015443f8ad1a678bceed57d', 'request_id': 'req-6659ae26-9796-4439-8715-883ef3f24901'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.045391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] The token cache is being invalidate because project 1fbc13137015443f8ad1a678bceed57d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:04.045717 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.049787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.049787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1271/2537] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:03 2022] DELETE /identity/v3/projects/1fbc13137015443f8ad1a678bceed57d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:04.051801 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.053149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:04.109244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dde614467d0457798e9b22f2ec33fc5'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.115926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dde614467d0457798e9b22f2ec33fc5'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.116414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659ae26-9796-4439-8715-883ef3f24901 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.121077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.121955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.122347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.122658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] PATH_INFO: `/v3/users/413eb745feae43d785f6cfb26d2956d9` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.123852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] RBAC: Authorizing `identity:delete_user(user_id=413eb745feae43d785f6cfb26d2956d9)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.125747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.230327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dde614467d0457798e9b22f2ec33fc5'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.231039 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dde614467d0457798e9b22f2ec33fc5'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.231652 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dde614467d0457798e9b22f2ec33fc5'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.250621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1267/2538] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:03 2022] DELETE /identity/v3/users/5dde614467d0457798e9b22f2ec33fc5 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:04.251439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.254647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:04.264582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:04.321260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0b9c-3e79-4ff1-9cd5-3331abe9e212 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.327149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.328440 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.328440 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.328650 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] PATH_INFO: `/v3/projects/69eec38386e648d8904185a634a50775` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.329717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69eec38386e648d8904185a634a50775)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.332020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.382841 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413eb745feae43d785f6cfb26d2956d9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.390255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413eb745feae43d785f6cfb26d2956d9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.390980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413eb745feae43d785f6cfb26d2956d9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.449531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69eec38386e648d8904185a634a50775', 'request_id': 'req-2f6fb544-d470-499d-9388-49c3c4904a20'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.455543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413eb745feae43d785f6cfb26d2956d9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.455854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69eec38386e648d8904185a634a50775', 'request_id': 'req-2f6fb544-d470-499d-9388-49c3c4904a20'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.456394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-01ec649b-f1ba-4c06-b03c-0cded3b0699b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413eb745feae43d785f6cfb26d2956d9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.462825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69eec38386e648d8904185a634a50775', 'request_id': 'req-2f6fb544-d470-499d-9388-49c3c4904a20'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.477385 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1272/2539] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:04 2022] DELETE /identity/v3/users/413eb745feae43d785f6cfb26d2956d9 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:04.547617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] The token cache is being invalidate because project 69eec38386e648d8904185a634a50775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:04.547915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.548562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2f6fb544-d470-499d-9388-49c3c4904a20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.550132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1268/2540] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:04 2022] DELETE /identity/v3/projects/69eec38386e648d8904185a634a50775 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:04.553383 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:04.556450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5061f-046f-468d-b30d-bbcc1b2dfec0 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.556790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5061f-046f-468d-b30d-bbcc1b2dfec0 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.557082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5061f-046f-468d-b30d-bbcc1b2dfec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.582623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-67e5061f-046f-468d-b30d-bbcc1b2dfec0 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:04.582623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-67e5061f-046f-468d-b30d-bbcc1b2dfec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:04.642414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1273/2541] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:04.651175 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03775f0-0323-4d2f-82b5-ddeb615f2b45 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.651462 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03775f0-0323-4d2f-82b5-ddeb615f2b45 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.651741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03775f0-0323-4d2f-82b5-ddeb615f2b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.670446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a03775f0-0323-4d2f-82b5-ddeb615f2b45 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:04.671686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a03775f0-0323-4d2f-82b5-ddeb615f2b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:04.679669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1269/2542] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:04.691696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5061f-046f-468d-b30d-bbcc1b2dfec0 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.695438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5061f-046f-468d-b30d-bbcc1b2dfec0 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.700320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d70df-9adc-4eee-82ab-97bfac998153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.701059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d70df-9adc-4eee-82ab-97bfac998153 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.701346 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d70df-9adc-4eee-82ab-97bfac998153 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.701625 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d70df-9adc-4eee-82ab-97bfac998153 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.702349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d70df-9adc-4eee-82ab-97bfac998153 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:04.702690 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d70df-9adc-4eee-82ab-97bfac998153 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.704337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d70df-9adc-4eee-82ab-97bfac998153 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.716748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1274/2543] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:04.721340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03775f0-0323-4d2f-82b5-ddeb615f2b45 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.745474 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03775f0-0323-4d2f-82b5-ddeb615f2b45 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.753192 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d70df-9adc-4eee-82ab-97bfac998153 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.753464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8c8e9-17a4-475b-9f81-f3fd5c3ce0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.754309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8c8e9-17a4-475b-9f81-f3fd5c3ce0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.754694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8c8e9-17a4-475b-9f81-f3fd5c3ce0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.755046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8c8e9-17a4-475b-9f81-f3fd5c3ce0e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.755820 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8c8e9-17a4-475b-9f81-f3fd5c3ce0e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.755957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-382d70df-9adc-4eee-82ab-97bfac998153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:04.757980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8c8e9-17a4-475b-9f81-f3fd5c3ce0e5 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.770682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d70df-9adc-4eee-82ab-97bfac998153 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.778714 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.779638 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.779861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.780154 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] PATH_INFO: `/v3/projects/b7cc96c0f75c43e09cab01b33520d441` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.781853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7cc96c0f75c43e09cab01b33520d441)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.783852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.899519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1270/2544] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:04 2022] POST /identity/v3/projects => generated 369 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:04.903526 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8c8e9-17a4-475b-9f81-f3fd5c3ce0e5 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.913336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7cc96c0f75c43e09cab01b33520d441', 'request_id': 'req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.921518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7cc96c0f75c43e09cab01b33520d441', 'request_id': 'req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.927718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7cc96c0f75c43e09cab01b33520d441', 'request_id': 'req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:04.965242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8c8e9-17a4-475b-9f81-f3fd5c3ce0e5 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.970013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519992c-4b8f-4d9c-9d1e-79e05702f2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.970699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519992c-4b8f-4d9c-9d1e-79e05702f2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.970984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519992c-4b8f-4d9c-9d1e-79e05702f2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.971263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519992c-4b8f-4d9c-9d1e-79e05702f2b4 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.971957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519992c-4b8f-4d9c-9d1e-79e05702f2b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.973887 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519992c-4b8f-4d9c-9d1e-79e05702f2b4 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.060691 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] The token cache is being invalidate because project b7cc96c0f75c43e09cab01b33520d441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:05.061547 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:05.062066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:05.063757 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1275/2545] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:04 2022] DELETE /identity/v3/projects/b7cc96c0f75c43e09cab01b33520d441 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:05.064872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.066201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.085465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1271/2546] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:04 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:05.087596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a21df-a88a-43bb-ac09-96f1d5389d85 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.092103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a21df-a88a-43bb-ac09-96f1d5389d85 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.092103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a21df-a88a-43bb-ac09-96f1d5389d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.113664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-958a21df-a88a-43bb-ac09-96f1d5389d85 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:05.113664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-958a21df-a88a-43bb-ac09-96f1d5389d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.127974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1272/2547] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:05.127974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a21df-a88a-43bb-ac09-96f1d5389d85 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.127974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-958a21df-a88a-43bb-ac09-96f1d5389d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.141886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a21df-a88a-43bb-ac09-96f1d5389d85 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.142001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95e563-22ef-4fed-a7a4-7c6d09b23a69 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.142795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25802f-b9d5-4961-9bfe-47bbf20ec0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.143455 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25802f-b9d5-4961-9bfe-47bbf20ec0ca admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.143909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25802f-b9d5-4961-9bfe-47bbf20ec0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.144348 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25802f-b9d5-4961-9bfe-47bbf20ec0ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.145037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.145349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25802f-b9d5-4961-9bfe-47bbf20ec0ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.145930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.146211 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.146682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] PATH_INFO: `/v3/projects/2c096761bee84569ac2ac78d908fd752` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.146907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25802f-b9d5-4961-9bfe-47bbf20ec0ca admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.147881 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c096761bee84569ac2ac78d908fd752)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.150572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.161289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1273/2548] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:05.161289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3ae90-f9ba-4d41-b969-a85c4253a5af None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.161289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3ae90-f9ba-4d41-b969-a85c4253a5af None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.161289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3ae90-f9ba-4d41-b969-a85c4253a5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.180626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-94b3ae90-f9ba-4d41-b969-a85c4253a5af None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:05.182112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-94b3ae90-f9ba-4d41-b969-a85c4253a5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.191098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1274/2549] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:05.191912 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3ae90-f9ba-4d41-b969-a85c4253a5af None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.201486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3ae90-f9ba-4d41-b969-a85c4253a5af None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.205919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79547601-7e1e-4f17-a23d-d06e0971f647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.206590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79547601-7e1e-4f17-a23d-d06e0971f647 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.206868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79547601-7e1e-4f17-a23d-d06e0971f647 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.207199 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79547601-7e1e-4f17-a23d-d06e0971f647 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.207926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79547601-7e1e-4f17-a23d-d06e0971f647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.209504 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79547601-7e1e-4f17-a23d-d06e0971f647 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.219925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1275/2550] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:05.220946 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79547601-7e1e-4f17-a23d-d06e0971f647 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.230689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79547601-7e1e-4f17-a23d-d06e0971f647 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.235369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6924f-a395-4724-89c9-4c75a416bbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.236085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6924f-a395-4724-89c9-4c75a416bbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.236361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6924f-a395-4724-89c9-4c75a416bbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.236628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6924f-a395-4724-89c9-4c75a416bbf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.237344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6924f-a395-4724-89c9-4c75a416bbf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:05.237644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6924f-a395-4724-89c9-4c75a416bbf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.239297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6924f-a395-4724-89c9-4c75a416bbf8 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.251727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1276/2551] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:05.252563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6924f-a395-4724-89c9-4c75a416bbf8 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.262392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6924f-a395-4724-89c9-4c75a416bbf8 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.270384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c096761bee84569ac2ac78d908fd752', 'request_id': 'req-c9bedebc-aefb-49da-aeb5-da84ab041448'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:05.277161 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c096761bee84569ac2ac78d908fd752', 'request_id': 'req-c9bedebc-aefb-49da-aeb5-da84ab041448'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:05.282736 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c096761bee84569ac2ac78d908fd752', 'request_id': 'req-c9bedebc-aefb-49da-aeb5-da84ab041448'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:05.316098 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e014e8-daea-4bef-89f5-ad4ae347df83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.316982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e014e8-daea-4bef-89f5-ad4ae347df83 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.317260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e014e8-daea-4bef-89f5-ad4ae347df83 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.317533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e014e8-daea-4bef-89f5-ad4ae347df83 admin admin] PATH_INFO: `/v3/projects/7596aa36b16d45a0957a306d19a56113/users/6b92ec17aad046c489c0c7c7c0318a6f/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.333049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e014e8-daea-4bef-89f5-ad4ae347df83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7596aa36b16d45a0957a306d19a56113, user_id=6b92ec17aad046c489c0c7c7c0318a6f, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.335749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e014e8-daea-4bef-89f5-ad4ae347df83 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.352610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] The token cache is being invalidate because project 2c096761bee84569ac2ac78d908fd752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:05.352899 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:05.353702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:05.358980 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1276/2552] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:05 2022] DELETE /identity/v3/projects/2c096761bee84569ac2ac78d908fd752 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:05.359756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.361065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.374214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bedebc-aefb-49da-aeb5-da84ab041448 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.378996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b75bc8-a6a9-4c65-9d92-802bdfc1609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.379647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b75bc8-a6a9-4c65-9d92-802bdfc1609b admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.379936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b75bc8-a6a9-4c65-9d92-802bdfc1609b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.380214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b75bc8-a6a9-4c65-9d92-802bdfc1609b admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.380902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b75bc8-a6a9-4c65-9d92-802bdfc1609b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.382842 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b75bc8-a6a9-4c65-9d92-802bdfc1609b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.472294 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1277/2553] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:05 2022] PUT /identity/v3/projects/7596aa36b16d45a0957a306d19a56113/users/6b92ec17aad046c489c0c7c7c0318a6f/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:05.473723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532393d-6aa6-4668-8282-b0b35b19e576 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.474012 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532393d-6aa6-4668-8282-b0b35b19e576 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.474392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532393d-6aa6-4668-8282-b0b35b19e576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.491856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e532393d-6aa6-4668-8282-b0b35b19e576 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:05.493609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e532393d-6aa6-4668-8282-b0b35b19e576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.540455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1277/2554] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:05.540455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05841f8-7250-4e67-aa8c-2b152045a01f None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.540455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05841f8-7250-4e67-aa8c-2b152045a01f None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.540455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05841f8-7250-4e67-aa8c-2b152045a01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.555532 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1278/2555] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:05.556550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532393d-6aa6-4668-8282-b0b35b19e576 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.568688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a05841f8-7250-4e67-aa8c-2b152045a01f None None] MFA Rules not processed for user `6b92ec17aad046c489c0c7c7c0318a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:05.569935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a05841f8-7250-4e67-aa8c-2b152045a01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.575717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532393d-6aa6-4668-8282-b0b35b19e576 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.637831 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2eccf2-5244-47f2-b6cf-c17229fea908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.638147 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1278/2556] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:05.638615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2eccf2-5244-47f2-b6cf-c17229fea908 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.639066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2eccf2-5244-47f2-b6cf-c17229fea908 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.639477 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2eccf2-5244-47f2-b6cf-c17229fea908 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.640181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2eccf2-5244-47f2-b6cf-c17229fea908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.643505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2eccf2-5244-47f2-b6cf-c17229fea908 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.644764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda33f5-bc5c-4e3e-8c8d-0e59aeb67173 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.645058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda33f5-bc5c-4e3e-8c8d-0e59aeb67173 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.645342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda33f5-bc5c-4e3e-8c8d-0e59aeb67173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.664568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-fdda33f5-bc5c-4e3e-8c8d-0e59aeb67173 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:05.666067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-fdda33f5-bc5c-4e3e-8c8d-0e59aeb67173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.674552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1279/2557] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:05.674552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda33f5-bc5c-4e3e-8c8d-0e59aeb67173 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.678815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-fdda33f5-bc5c-4e3e-8c8d-0e59aeb67173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.736518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda33f5-bc5c-4e3e-8c8d-0e59aeb67173 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.740243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1279/2558] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/users => generated 291 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:05.740243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2eccf2-5244-47f2-b6cf-c17229fea908 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.743759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec8fbb-e19d-44eb-947c-d9004a6f9962 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e12361da5f849ee97607ca68ee6ef10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.744421 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec8fbb-e19d-44eb-947c-d9004a6f9962 service glance] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.744704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec8fbb-e19d-44eb-947c-d9004a6f9962 service glance] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.747720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec8fbb-e19d-44eb-947c-d9004a6f9962 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.750090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2eccf2-5244-47f2-b6cf-c17229fea908 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.754866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88941250-7a9f-4620-be58-e216db8b720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.755590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88941250-7a9f-4620-be58-e216db8b720e admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.755878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88941250-7a9f-4620-be58-e216db8b720e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.756173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88941250-7a9f-4620-be58-e216db8b720e admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.757005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88941250-7a9f-4620-be58-e216db8b720e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:05.757313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88941250-7a9f-4620-be58-e216db8b720e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.757418 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec8fbb-e19d-44eb-947c-d9004a6f9962 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.759444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec8fbb-e19d-44eb-947c-d9004a6f9962 service glance] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.759568 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88941250-7a9f-4620-be58-e216db8b720e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.768687 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1280/2559] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:05.769497 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec8fbb-e19d-44eb-947c-d9004a6f9962 service glance] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.771929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1280/2560] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:05.776255 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88941250-7a9f-4620-be58-e216db8b720e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.782764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec8fbb-e19d-44eb-947c-d9004a6f9962 service glance] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.786265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88941250-7a9f-4620-be58-e216db8b720e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.787723 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b63d9-07d8-4efa-bcac-44fe29947b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.788390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b63d9-07d8-4efa-bcac-44fe29947b40 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.788674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b63d9-07d8-4efa-bcac-44fe29947b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.788952 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b63d9-07d8-4efa-bcac-44fe29947b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.789763 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b63d9-07d8-4efa-bcac-44fe29947b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.791380 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b63d9-07d8-4efa-bcac-44fe29947b40 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.794465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14c85a-fea8-4e02-9e5e-1de4901a3825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.795741 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14c85a-fea8-4e02-9e5e-1de4901a3825 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.796246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14c85a-fea8-4e02-9e5e-1de4901a3825 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.796740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14c85a-fea8-4e02-9e5e-1de4901a3825 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.797659 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14c85a-fea8-4e02-9e5e-1de4901a3825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.799879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14c85a-fea8-4e02-9e5e-1de4901a3825 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.802692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1281/2561] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:05.807444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b63d9-07d8-4efa-bcac-44fe29947b40 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.817582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b63d9-07d8-4efa-bcac-44fe29947b40 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.823066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35cfcc-0f30-4f98-a44c-cf93efbef6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.823738 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35cfcc-0f30-4f98-a44c-cf93efbef6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.826729 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35cfcc-0f30-4f98-a44c-cf93efbef6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.827071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35cfcc-0f30-4f98-a44c-cf93efbef6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.827799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35cfcc-0f30-4f98-a44c-cf93efbef6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.829416 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35cfcc-0f30-4f98-a44c-cf93efbef6d0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.843388 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1282/2562] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:05.844316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35cfcc-0f30-4f98-a44c-cf93efbef6d0 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.854850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35cfcc-0f30-4f98-a44c-cf93efbef6d0 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.859649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a7818-d25f-49bf-acd0-4f67a35e98c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.860505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a7818-d25f-49bf-acd0-4f67a35e98c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.860788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a7818-d25f-49bf-acd0-4f67a35e98c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.861063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a7818-d25f-49bf-acd0-4f67a35e98c7 admin admin] PATH_INFO: `/v3/projects/076218f41775459b9e0b8663029f3d57/users/804a6ed9d9e74b6b91e988b333e66961/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.877362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a7818-d25f-49bf-acd0-4f67a35e98c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076218f41775459b9e0b8663029f3d57, user_id=804a6ed9d9e74b6b91e988b333e66961, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.880109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a7818-d25f-49bf-acd0-4f67a35e98c7 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.919208 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1281/2563] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:05.921471 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:05.923491 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14c85a-fea8-4e02-9e5e-1de4901a3825 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.970448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1283/2564] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:05 2022] PUT /identity/v3/projects/076218f41775459b9e0b8663029f3d57/users/804a6ed9d9e74b6b91e988b333e66961/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:05.982700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6443a-a377-4664-a688-7b820bd1abe2 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.983078 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6443a-a377-4664-a688-7b820bd1abe2 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.983350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6443a-a377-4664-a688-7b820bd1abe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.990938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14c85a-fea8-4e02-9e5e-1de4901a3825 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.015573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-02b6443a-a377-4664-a688-7b820bd1abe2 None None] MFA Rules not processed for user `804a6ed9d9e74b6b91e988b333e66961`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:06.016791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-02b6443a-a377-4664-a688-7b820bd1abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:06.057593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c48ac-b10d-4366-9415-d27a20d1d1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.058820 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c48ac-b10d-4366-9415-d27a20d1d1fe admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.059099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c48ac-b10d-4366-9415-d27a20d1d1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.059390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c48ac-b10d-4366-9415-d27a20d1d1fe admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.060829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c48ac-b10d-4366-9415-d27a20d1d1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.062827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c48ac-b10d-4366-9415-d27a20d1d1fe admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.087164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1284/2565] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:06.091505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6443a-a377-4664-a688-7b820bd1abe2 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.101408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6443a-a377-4664-a688-7b820bd1abe2 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.106524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9b7b9-974a-4aac-bc05-feb0dc5a0cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.111425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9b7b9-974a-4aac-bc05-feb0dc5a0cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.111425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9b7b9-974a-4aac-bc05-feb0dc5a0cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.111425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9b7b9-974a-4aac-bc05-feb0dc5a0cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.111425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9b7b9-974a-4aac-bc05-feb0dc5a0cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.111425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9b7b9-974a-4aac-bc05-feb0dc5a0cf1 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.158791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:06.165102 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1282/2566] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/users => generated 290 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:06.169681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c48ac-b10d-4366-9415-d27a20d1d1fe admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.178846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c48ac-b10d-4366-9415-d27a20d1d1fe admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.184010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9145bf9-8bea-48d7-9d31-f812ec02af44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.184637 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9145bf9-8bea-48d7-9d31-f812ec02af44 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.184915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9145bf9-8bea-48d7-9d31-f812ec02af44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.185195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9145bf9-8bea-48d7-9d31-f812ec02af44 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.186012 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9145bf9-8bea-48d7-9d31-f812ec02af44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.187655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9145bf9-8bea-48d7-9d31-f812ec02af44 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.197289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1283/2567] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:06.201079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9145bf9-8bea-48d7-9d31-f812ec02af44 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.209970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9145bf9-8bea-48d7-9d31-f812ec02af44 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.215109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9157ca-e725-496a-b029-9ea69a848040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.215722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9157ca-e725-496a-b029-9ea69a848040 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.216007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9157ca-e725-496a-b029-9ea69a848040 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.216281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9157ca-e725-496a-b029-9ea69a848040 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.216975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9157ca-e725-496a-b029-9ea69a848040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.219642 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9157ca-e725-496a-b029-9ea69a848040 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.225206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1285/2568] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:06 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:06.226081 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9b7b9-974a-4aac-bc05-feb0dc5a0cf1 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.231710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1284/2569] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:06.235790 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9157ca-e725-496a-b029-9ea69a848040 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.245999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9157ca-e725-496a-b029-9ea69a848040 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.293793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9b7b9-974a-4aac-bc05-feb0dc5a0cf1 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.298582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ba970-08a1-48b9-a1fa-8c468d138757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.299279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ba970-08a1-48b9-a1fa-8c468d138757 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.299561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ba970-08a1-48b9-a1fa-8c468d138757 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.299864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ba970-08a1-48b9-a1fa-8c468d138757 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.300541 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ba970-08a1-48b9-a1fa-8c468d138757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.302462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ba970-08a1-48b9-a1fa-8c468d138757 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.303007 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef28ded-1366-4162-adaa-32507de48a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.303954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef28ded-1366-4162-adaa-32507de48a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.304323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef28ded-1366-4162-adaa-32507de48a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.304613 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef28ded-1366-4162-adaa-32507de48a6e admin admin] PATH_INFO: `/v3/projects/8af624478ff24472b64d223034c2434b/users/9bf65da96422444eaffeea9ee293a8a0/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.319293 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef28ded-1366-4162-adaa-32507de48a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8af624478ff24472b64d223034c2434b, user_id=9bf65da96422444eaffeea9ee293a8a0, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.321968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef28ded-1366-4162-adaa-32507de48a6e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.462204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1285/2570] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:06 2022] PUT /identity/v3/projects/8af624478ff24472b64d223034c2434b/users/9bf65da96422444eaffeea9ee293a8a0/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:06.464700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1286/2571] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:06 2022] POST /identity/v3/users => generated 296 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:06.466518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef28ded-1366-4162-adaa-32507de48a6e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.473915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87e4de-476d-4a66-b650-647c8d331324 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.474238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87e4de-476d-4a66-b650-647c8d331324 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.474568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87e4de-476d-4a66-b650-647c8d331324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.501212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-4c87e4de-476d-4a66-b650-647c8d331324 None None] MFA Rules not processed for user `9bf65da96422444eaffeea9ee293a8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:06.502373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4c87e4de-476d-4a66-b650-647c8d331324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:06.527980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef28ded-1366-4162-adaa-32507de48a6e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.532911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dd20a-91ff-4d5a-bdc7-06af5b6bc621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.533573 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dd20a-91ff-4d5a-bdc7-06af5b6bc621 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.533854 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dd20a-91ff-4d5a-bdc7-06af5b6bc621 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.534129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dd20a-91ff-4d5a-bdc7-06af5b6bc621 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.534975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dd20a-91ff-4d5a-bdc7-06af5b6bc621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.536566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dd20a-91ff-4d5a-bdc7-06af5b6bc621 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.547235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1286/2572] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:06.551193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dd20a-91ff-4d5a-bdc7-06af5b6bc621 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.561675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dd20a-91ff-4d5a-bdc7-06af5b6bc621 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.566664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbad176-deda-490b-ac3f-8d924c2ce15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.567558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad176-deda-490b-ac3f-8d924c2ce15e admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.567832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad176-deda-490b-ac3f-8d924c2ce15e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.568115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad176-deda-490b-ac3f-8d924c2ce15e admin admin] PATH_INFO: `/v3/projects/fb752141ce8e491799d9f939a83db53c/users/9ac82e6e46d24fec939d47f181cbf3e3/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.571965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1287/2573] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:06.576195 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87e4de-476d-4a66-b650-647c8d331324 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.584055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbad176-deda-490b-ac3f-8d924c2ce15e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb752141ce8e491799d9f939a83db53c, user_id=9ac82e6e46d24fec939d47f181cbf3e3, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.586872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbad176-deda-490b-ac3f-8d924c2ce15e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.587819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87e4de-476d-4a66-b650-647c8d331324 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.592953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f62a7e-8c95-45e9-9242-5cbac6d0ade9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.593732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f62a7e-8c95-45e9-9242-5cbac6d0ade9 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.594085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f62a7e-8c95-45e9-9242-5cbac6d0ade9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.594372 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f62a7e-8c95-45e9-9242-5cbac6d0ade9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.595112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f62a7e-8c95-45e9-9242-5cbac6d0ade9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.597010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f62a7e-8c95-45e9-9242-5cbac6d0ade9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.685150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1287/2574] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:06 2022] PUT /identity/v3/projects/fb752141ce8e491799d9f939a83db53c/users/9ac82e6e46d24fec939d47f181cbf3e3/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:06.689384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbad176-deda-490b-ac3f-8d924c2ce15e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.747024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1288/2575] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:06 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:06.747984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f62a7e-8c95-45e9-9242-5cbac6d0ade9 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.757091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbad176-deda-490b-ac3f-8d924c2ce15e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.813409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f62a7e-8c95-45e9-9242-5cbac6d0ade9 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.818442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bda99d-3a8f-41e8-9bc5-c8bd6c15901b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.819360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bda99d-3a8f-41e8-9bc5-c8bd6c15901b admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.819503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bda99d-3a8f-41e8-9bc5-c8bd6c15901b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.819712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bda99d-3a8f-41e8-9bc5-c8bd6c15901b admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.820417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bda99d-3a8f-41e8-9bc5-c8bd6c15901b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.822417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bda99d-3a8f-41e8-9bc5-c8bd6c15901b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.835229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7be467-f30f-44c1-ac70-75e1bace925a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.836127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7be467-f30f-44c1-ac70-75e1bace925a admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.836558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7be467-f30f-44c1-ac70-75e1bace925a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.837250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7be467-f30f-44c1-ac70-75e1bace925a admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.837945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7be467-f30f-44c1-ac70-75e1bace925a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.840192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7be467-f30f-44c1-ac70-75e1bace925a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.853590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1288/2576] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:06.860803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7be467-f30f-44c1-ac70-75e1bace925a admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.871671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7be467-f30f-44c1-ac70-75e1bace925a admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:06.876381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1760f-4a75-454e-9ea5-7a56de0c134c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:06.877474 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1760f-4a75-454e-9ea5-7a56de0c134c admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.877639 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1760f-4a75-454e-9ea5-7a56de0c134c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.877767 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1760f-4a75-454e-9ea5-7a56de0c134c admin admin] PATH_INFO: `/v3/projects/fb752141ce8e491799d9f939a83db53c/users/9ac82e6e46d24fec939d47f181cbf3e3/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.879805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1760f-4a75-454e-9ea5-7a56de0c134c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb752141ce8e491799d9f939a83db53c, user_id=9ac82e6e46d24fec939d47f181cbf3e3, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:06.882434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1760f-4a75-454e-9ea5-7a56de0c134c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:06.894186 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1289/2577] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:06 2022] PUT /identity/v3/projects/fb752141ce8e491799d9f939a83db53c/users/9ac82e6e46d24fec939d47f181cbf3e3/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:06.904222 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca46a7-3726-4077-8bce-cc59aebbf6de None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:06.904508 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca46a7-3726-4077-8bce-cc59aebbf6de None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:06.904786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca46a7-3726-4077-8bce-cc59aebbf6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:06.938641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d0ca46a7-3726-4077-8bce-cc59aebbf6de None None] MFA Rules not processed for user `9ac82e6e46d24fec939d47f181cbf3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:06.939959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d0ca46a7-3726-4077-8bce-cc59aebbf6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:06.962015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1289/2578] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:06 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:06.963109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bda99d-3a8f-41e8-9bc5-c8bd6c15901b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:06.973620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bda99d-3a8f-41e8-9bc5-c8bd6c15901b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:07.021123 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1290/2579] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:07.036343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca46a7-3726-4077-8bce-cc59aebbf6de None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:07.038017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d0ca46a7-3726-4077-8bce-cc59aebbf6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:07.046066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99940ca7-5259-41e9-930b-75455519b032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:07.046856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99940ca7-5259-41e9-930b-75455519b032 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:07.047141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99940ca7-5259-41e9-930b-75455519b032 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:07.050143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99940ca7-5259-41e9-930b-75455519b032 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:07.050143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99940ca7-5259-41e9-930b-75455519b032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:07.050143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99940ca7-5259-41e9-930b-75455519b032 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:07.063810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1290/2580] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:07.064843 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99940ca7-5259-41e9-930b-75455519b032 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:07.075973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99940ca7-5259-41e9-930b-75455519b032 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:07.081142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e532de-339d-427c-a952-af3ede7689e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:07.082046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e532de-339d-427c-a952-af3ede7689e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:07.082336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e532de-339d-427c-a952-af3ede7689e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:07.082671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e532de-339d-427c-a952-af3ede7689e8 admin admin] PATH_INFO: `/v3/projects/c4c0746f26684779be2b4ca7b3c78ed5/users/19cb75aa9eaa4afbb0f0667996c3d95f/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:07.099495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e532de-339d-427c-a952-af3ede7689e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c0746f26684779be2b4ca7b3c78ed5, user_id=19cb75aa9eaa4afbb0f0667996c3d95f, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:07.102884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e532de-339d-427c-a952-af3ede7689e8 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:07.110683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:07.116779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca46a7-3726-4077-8bce-cc59aebbf6de None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:07.122200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c18856-2271-45b6-849c-9cc36968cfdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:07.122574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c18856-2271-45b6-849c-9cc36968cfdb service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:07.122922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c18856-2271-45b6-849c-9cc36968cfdb service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:07.123260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c18856-2271-45b6-849c-9cc36968cfdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:07.134829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c18856-2271-45b6-849c-9cc36968cfdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:07.136397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c18856-2271-45b6-849c-9cc36968cfdb service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:07.146539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1291/2581] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:07.186613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1291/2582] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:07 2022] PUT /identity/v3/projects/c4c0746f26684779be2b4ca7b3c78ed5/users/19cb75aa9eaa4afbb0f0667996c3d95f/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:07.191104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c18856-2271-45b6-849c-9cc36968cfdb service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:07.204376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c18856-2271-45b6-849c-9cc36968cfdb service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:07.267743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cacc30-082b-433e-aaa5-d0fd7a738a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:07.268502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cacc30-082b-433e-aaa5-d0fd7a738a87 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:07.268855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cacc30-082b-433e-aaa5-d0fd7a738a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:07.269202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cacc30-082b-433e-aaa5-d0fd7a738a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:07.270035 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cacc30-082b-433e-aaa5-d0fd7a738a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:07.271918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cacc30-082b-433e-aaa5-d0fd7a738a87 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:07.284144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1292/2583] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:07.288214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e532de-339d-427c-a952-af3ede7689e8 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:07.300222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e532de-339d-427c-a952-af3ede7689e8 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:07.310039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b99001-29d5-459e-b53b-260156f18d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:07.310992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99001-29d5-459e-b53b-260156f18d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:07.311278 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99001-29d5-459e-b53b-260156f18d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:07.311558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99001-29d5-459e-b53b-260156f18d43 admin admin] PATH_INFO: `/v3/projects/c4c0746f26684779be2b4ca7b3c78ed5/users/19cb75aa9eaa4afbb0f0667996c3d95f/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:07.313582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b99001-29d5-459e-b53b-260156f18d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c0746f26684779be2b4ca7b3c78ed5, user_id=19cb75aa9eaa4afbb0f0667996c3d95f, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:07.316268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b99001-29d5-459e-b53b-260156f18d43 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:07.328136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1292/2584] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:07 2022] PUT /identity/v3/projects/c4c0746f26684779be2b4ca7b3c78ed5/users/19cb75aa9eaa4afbb0f0667996c3d95f/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:07.351563 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0f147-ebab-4e07-bc8a-35fe13ce8c62 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:07.352341 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0f147-ebab-4e07-bc8a-35fe13ce8c62 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:07.352806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0f147-ebab-4e07-bc8a-35fe13ce8c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:07.384159 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-2bc0f147-ebab-4e07-bc8a-35fe13ce8c62 None None] MFA Rules not processed for user `19cb75aa9eaa4afbb0f0667996c3d95f`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:07.385550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2bc0f147-ebab-4e07-bc8a-35fe13ce8c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:07.417911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b99001-29d5-459e-b53b-260156f18d43 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:07.433353 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:07.468471 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1293/2585] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:07.479881 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0f147-ebab-4e07-bc8a-35fe13ce8c62 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:07.481310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2bc0f147-ebab-4e07-bc8a-35fe13ce8c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:07.491071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b99001-29d5-459e-b53b-260156f18d43 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:07.495370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72e0fa-dbd2-481f-8597-d1e94654758f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:07.496276 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e0fa-dbd2-481f-8597-d1e94654758f service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:07.496557 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e0fa-dbd2-481f-8597-d1e94654758f service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:07.496834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e0fa-dbd2-481f-8597-d1e94654758f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:07.499483 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0f147-ebab-4e07-bc8a-35fe13ce8c62 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:07.503395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77479906-ca16-4a02-87de-d87375f55a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:07.504105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77479906-ca16-4a02-87de-d87375f55a6e service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:07.504393 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77479906-ca16-4a02-87de-d87375f55a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:07.504674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77479906-ca16-4a02-87de-d87375f55a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:07.514581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77479906-ca16-4a02-87de-d87375f55a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:07.516089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77479906-ca16-4a02-87de-d87375f55a6e service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:07.526500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1294/2586] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:07.560524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72e0fa-dbd2-481f-8597-d1e94654758f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:07.561976 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72e0fa-dbd2-481f-8597-d1e94654758f service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:07.572779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1293/2587] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:07.888604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:08.218302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77479906-ca16-4a02-87de-d87375f55a6e service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:08.296472 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:08.310925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77479906-ca16-4a02-87de-d87375f55a6e service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:08.316485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:08.317374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:08.317683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:08.317928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] PATH_INFO: `/v3/users/9bf65da96422444eaffeea9ee293a8a0` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:08.319193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf65da96422444eaffeea9ee293a8a0)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:08.321715 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:08.506559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:08.555718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf65da96422444eaffeea9ee293a8a0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:08.562542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf65da96422444eaffeea9ee293a8a0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:08.615731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf65da96422444eaffeea9ee293a8a0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:08.616357 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf65da96422444eaffeea9ee293a8a0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:08.616977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf65da96422444eaffeea9ee293a8a0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:08.636773 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1295/2588] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:08 2022] DELETE /identity/v3/users/9bf65da96422444eaffeea9ee293a8a0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:08.641308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58239e-1e54-4d12-aee7-f34ba4d33e19 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:08.641667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58239e-1e54-4d12-aee7-f34ba4d33e19 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:08.642038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58239e-1e54-4d12-aee7-f34ba4d33e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:08.659792 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8b58239e-1e54-4d12-aee7-f34ba4d33e19 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:08.661135 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8b58239e-1e54-4d12-aee7-f34ba4d33e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:08.732054 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1294/2589] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:08.743426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:08.745070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:08.812694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73de169-3579-42ca-9eb0-dcbae703dcb5 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:08.816302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d38c1-ca4a-4448-a19c-4acfdd08cbd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:08.816955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d38c1-ca4a-4448-a19c-4acfdd08cbd8 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:08.817237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d38c1-ca4a-4448-a19c-4acfdd08cbd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:08.817510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d38c1-ca4a-4448-a19c-4acfdd08cbd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:08.827422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d38c1-ca4a-4448-a19c-4acfdd08cbd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:08.829028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d38c1-ca4a-4448-a19c-4acfdd08cbd8 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:08.840344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1296/2590] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:08.848590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:09.114165 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:09.350000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58239e-1e54-4d12-aee7-f34ba4d33e19 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:09.354847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8b58239e-1e54-4d12-aee7-f34ba4d33e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:09.365128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58239e-1e54-4d12-aee7-f34ba4d33e19 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:09.372050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:09.373347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:09.373725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:09.374099 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] PATH_INFO: `/v3/projects/8af624478ff24472b64d223034c2434b` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:09.375497 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8af624478ff24472b64d223034c2434b)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:09.377575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:09.507675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8af624478ff24472b64d223034c2434b', 'request_id': 'req-ef0029b6-5cd2-42e3-9386-5975d99bb17d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:09.515131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8af624478ff24472b64d223034c2434b', 'request_id': 'req-ef0029b6-5cd2-42e3-9386-5975d99bb17d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:09.525286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8af624478ff24472b64d223034c2434b', 'request_id': 'req-ef0029b6-5cd2-42e3-9386-5975d99bb17d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:09.580773 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:09.609114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] The token cache is being invalidate because project 8af624478ff24472b64d223034c2434b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:09.609441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:09.610052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:09.611584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1295/2591] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:09 2022] DELETE /identity/v3/projects/8af624478ff24472b64d223034c2434b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:09.615738 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d38c1-ca4a-4448-a19c-4acfdd08cbd8 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:09.627311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d38c1-ca4a-4448-a19c-4acfdd08cbd8 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:09.662725 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:09.696574 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:09.697609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:09.698106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:09.698519 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] PATH_INFO: `/v3/users/19cb75aa9eaa4afbb0f0667996c3d95f` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:09.700118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19cb75aa9eaa4afbb0f0667996c3d95f)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:09.702246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:09.928320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19cb75aa9eaa4afbb0f0667996c3d95f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:09.937085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19cb75aa9eaa4afbb0f0667996c3d95f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:09.999746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:10.000426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19cb75aa9eaa4afbb0f0667996c3d95f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:10.001107 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19cb75aa9eaa4afbb0f0667996c3d95f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:10.001708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3c7fe9f6-53fd-4976-a408-fcfdb3b5a9a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19cb75aa9eaa4afbb0f0667996c3d95f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:10.019130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1297/2592] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:09 2022] DELETE /identity/v3/users/19cb75aa9eaa4afbb0f0667996c3d95f => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:10.447051 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:10.448688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:10.471928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0029b6-5cd2-42e3-9386-5975d99bb17d admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:10.520097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:10.533584 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:10.534511 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:10.534798 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:10.535074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] PATH_INFO: `/v3/projects/c4c0746f26684779be2b4ca7b3c78ed5` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:10.536256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4c0746f26684779be2b4ca7b3c78ed5)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:10.538165 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:10.652544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4c0746f26684779be2b4ca7b3c78ed5', 'request_id': 'req-56b674c8-212f-40a2-8d29-f0e4cec427b9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:10.659302 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4c0746f26684779be2b4ca7b3c78ed5', 'request_id': 'req-56b674c8-212f-40a2-8d29-f0e4cec427b9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:10.665357 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4c0746f26684779be2b4ca7b3c78ed5', 'request_id': 'req-56b674c8-212f-40a2-8d29-f0e4cec427b9'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:10.752333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] The token cache is being invalidate because project c4c0746f26684779be2b4ca7b3c78ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:10.753436 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:10.755266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-56b674c8-212f-40a2-8d29-f0e4cec427b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:10.758908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1296/2593] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:10 2022] DELETE /identity/v3/projects/c4c0746f26684779be2b4ca7b3c78ed5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:10.765113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2388e8-e7ae-476f-a264-55fb12cfb256 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:10.765979 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2388e8-e7ae-476f-a264-55fb12cfb256 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:10.765979 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2388e8-e7ae-476f-a264-55fb12cfb256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:10.784576 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-bf2388e8-e7ae-476f-a264-55fb12cfb256 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:10.785893 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-bf2388e8-e7ae-476f-a264-55fb12cfb256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:10.858682 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1298/2594] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:10.863019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e9fb-e9d0-47fa-8976-25acf45daf3d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:10.863307 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e9fb-e9d0-47fa-8976-25acf45daf3d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:10.863585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e9fb-e9d0-47fa-8976-25acf45daf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:10.882496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-f159e9fb-e9d0-47fa-8976-25acf45daf3d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:10.884367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f159e9fb-e9d0-47fa-8976-25acf45daf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:10.892783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1297/2595] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:10.896968 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2388e8-e7ae-476f-a264-55fb12cfb256 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:10.907063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2388e8-e7ae-476f-a264-55fb12cfb256 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:10.913698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee046fbd-c3f9-47d1-91da-97cc6eac3b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:10.914384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046fbd-c3f9-47d1-91da-97cc6eac3b78 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:10.914800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046fbd-c3f9-47d1-91da-97cc6eac3b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:10.915108 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046fbd-c3f9-47d1-91da-97cc6eac3b78 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:10.915853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee046fbd-c3f9-47d1-91da-97cc6eac3b78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:10.916204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee046fbd-c3f9-47d1-91da-97cc6eac3b78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:10.917974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee046fbd-c3f9-47d1-91da-97cc6eac3b78 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:10.937253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1299/2596] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:10.938949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159e9fb-e9d0-47fa-8976-25acf45daf3d None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:10.949111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159e9fb-e9d0-47fa-8976-25acf45daf3d None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:10.954334 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb95a1-392e-4d3b-98e7-46e96cbc4f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:10.955231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb95a1-392e-4d3b-98e7-46e96cbc4f7d admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:10.955617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb95a1-392e-4d3b-98e7-46e96cbc4f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:10.955996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb95a1-392e-4d3b-98e7-46e96cbc4f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:10.956793 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb95a1-392e-4d3b-98e7-46e96cbc4f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:10.958886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb95a1-392e-4d3b-98e7-46e96cbc4f7d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:11.051786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1298/2597] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:10 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:11.056351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee046fbd-c3f9-47d1-91da-97cc6eac3b78 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:11.066833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee046fbd-c3f9-47d1-91da-97cc6eac3b78 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:11.140763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97048921-e196-4506-b0ff-bade0c4bf8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:11.141230 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97048921-e196-4506-b0ff-bade0c4bf8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:11.141581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97048921-e196-4506-b0ff-bade0c4bf8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:11.142084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97048921-e196-4506-b0ff-bade0c4bf8e6 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:11.145623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97048921-e196-4506-b0ff-bade0c4bf8e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:11.145623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97048921-e196-4506-b0ff-bade0c4bf8e6 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:11.216779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1300/2598] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:11 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:11.221187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb95a1-392e-4d3b-98e7-46e96cbc4f7d admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:11.232408 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb95a1-392e-4d3b-98e7-46e96cbc4f7d admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:11.237605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b602868-5593-4b76-ac35-1e482a7db610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:11.238281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b602868-5593-4b76-ac35-1e482a7db610 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:11.238615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b602868-5593-4b76-ac35-1e482a7db610 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:11.238916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b602868-5593-4b76-ac35-1e482a7db610 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:11.239686 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b602868-5593-4b76-ac35-1e482a7db610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:11.241335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b602868-5593-4b76-ac35-1e482a7db610 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:11.251732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1299/2599] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:11.256026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97048921-e196-4506-b0ff-bade0c4bf8e6 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:11.267279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97048921-e196-4506-b0ff-bade0c4bf8e6 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:11.271970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b03259-4018-4d42-90c8-aa08b6fc0f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:11.272633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b03259-4018-4d42-90c8-aa08b6fc0f0a admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:11.272915 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b03259-4018-4d42-90c8-aa08b6fc0f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:11.273189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b03259-4018-4d42-90c8-aa08b6fc0f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:11.274625 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b03259-4018-4d42-90c8-aa08b6fc0f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:11.276225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b03259-4018-4d42-90c8-aa08b6fc0f0a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:11.286079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1301/2600] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:11.290688 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b602868-5593-4b76-ac35-1e482a7db610 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:11.302105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b602868-5593-4b76-ac35-1e482a7db610 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:11.306878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512c207-7057-4a4d-b7e8-189a0cc02f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:11.307782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512c207-7057-4a4d-b7e8-189a0cc02f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:11.308058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512c207-7057-4a4d-b7e8-189a0cc02f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:11.308327 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512c207-7057-4a4d-b7e8-189a0cc02f68 admin admin] PATH_INFO: `/v3/projects/ce1e92a4ddad48a0a2156db83def1660/users/859f6afc5ac74a9aaebaf73b72dcccc1/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:11.325814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512c207-7057-4a4d-b7e8-189a0cc02f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1e92a4ddad48a0a2156db83def1660, user_id=859f6afc5ac74a9aaebaf73b72dcccc1, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:11.328874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512c207-7057-4a4d-b7e8-189a0cc02f68 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:11.415592 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1300/2601] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:11 2022] PUT /identity/v3/projects/ce1e92a4ddad48a0a2156db83def1660/users/859f6afc5ac74a9aaebaf73b72dcccc1/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:11.428153 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747f0d3-1d1f-4da4-a913-e21aad933b76 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:11.428351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747f0d3-1d1f-4da4-a913-e21aad933b76 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:11.428581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747f0d3-1d1f-4da4-a913-e21aad933b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:11.462184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-8747f0d3-1d1f-4da4-a913-e21aad933b76 None None] MFA Rules not processed for user `859f6afc5ac74a9aaebaf73b72dcccc1`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:11.463453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8747f0d3-1d1f-4da4-a913-e21aad933b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:11.481865 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:11.535369 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1302/2602] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:11.549187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512c207-7057-4a4d-b7e8-189a0cc02f68 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:11.551140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4512c207-7057-4a4d-b7e8-189a0cc02f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:11.609073 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512c207-7057-4a4d-b7e8-189a0cc02f68 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:11.612895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3432f-beba-4615-98d9-9b36a37d9af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:11.613551 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3432f-beba-4615-98d9-9b36a37d9af1 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:11.613833 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3432f-beba-4615-98d9-9b36a37d9af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:11.614109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3432f-beba-4615-98d9-9b36a37d9af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:11.623384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3432f-beba-4615-98d9-9b36a37d9af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:11.624907 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3432f-beba-4615-98d9-9b36a37d9af1 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:11.634549 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1301/2603] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:16.041151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747f0d3-1d1f-4da4-a913-e21aad933b76 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:16.043145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8747f0d3-1d1f-4da4-a913-e21aad933b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:16.119573 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747f0d3-1d1f-4da4-a913-e21aad933b76 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:16.120166 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d36b29-a23c-4d0b-bb6c-2b2784d294ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:16.120829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d36b29-a23c-4d0b-bb6c-2b2784d294ba service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:16.121113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d36b29-a23c-4d0b-bb6c-2b2784d294ba service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:16.121391 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d36b29-a23c-4d0b-bb6c-2b2784d294ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:16.123353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-81d36b29-a23c-4d0b-bb6c-2b2784d294ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:16.197537 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d36b29-a23c-4d0b-bb6c-2b2784d294ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:16.199064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d36b29-a23c-4d0b-bb6c-2b2784d294ba service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:16.213672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1303/2604] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:19.474254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3432f-beba-4615-98d9-9b36a37d9af1 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:19.477198 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a1c3432f-beba-4615-98d9-9b36a37d9af1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:19.500116 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3432f-beba-4615-98d9-9b36a37d9af1 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:19.506409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:19.507815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:19.508376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:19.509035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] PATH_INFO: `/v3/users/804a6ed9d9e74b6b91e988b333e66961` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:19.513113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=804a6ed9d9e74b6b91e988b333e66961)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:19.516142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:19.550275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:19.809394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '804a6ed9d9e74b6b91e988b333e66961'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:19.820088 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '804a6ed9d9e74b6b91e988b333e66961'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:19.820442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '804a6ed9d9e74b6b91e988b333e66961'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:19.920345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '804a6ed9d9e74b6b91e988b333e66961'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:19.921550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '804a6ed9d9e74b6b91e988b333e66961'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:19.961483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1302/2605] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:19 2022] DELETE /identity/v3/users/804a6ed9d9e74b6b91e988b333e66961 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:19.966116 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557d0b1-a446-4598-a828-0cbc41270e02 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:19.966644 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557d0b1-a446-4598-a828-0cbc41270e02 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:19.966838 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557d0b1-a446-4598-a828-0cbc41270e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:19.988931 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-8557d0b1-a446-4598-a828-0cbc41270e02 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:19.990263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8557d0b1-a446-4598-a828-0cbc41270e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:20.067073 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1304/2606] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:20.077828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:20.079939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:20.180485 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d6bde-e1ee-44ff-9dee-be9f9371ea1b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:20.185238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f0fb8-7693-4b83-9a7b-283859f7c5da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:20.186214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f0fb8-7693-4b83-9a7b-283859f7c5da service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:20.186715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f0fb8-7693-4b83-9a7b-283859f7c5da service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:20.186998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f0fb8-7693-4b83-9a7b-283859f7c5da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:20.201780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f0fb8-7693-4b83-9a7b-283859f7c5da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:20.203337 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f0fb8-7693-4b83-9a7b-283859f7c5da service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:20.215737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1303/2607] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:20.320374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557d0b1-a446-4598-a828-0cbc41270e02 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:20.329990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8557d0b1-a446-4598-a828-0cbc41270e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:20.339971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557d0b1-a446-4598-a828-0cbc41270e02 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:20.346977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:20.348075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:20.348421 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:20.349010 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] PATH_INFO: `/v3/users/7f269cd38f734209ba0496cf70469608` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:20.350380 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f269cd38f734209ba0496cf70469608)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:20.356768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:20.358708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:20.577386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f269cd38f734209ba0496cf70469608'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:20.584087 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f269cd38f734209ba0496cf70469608'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:20.641840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f0fb8-7693-4b83-9a7b-283859f7c5da service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:20.643432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a47f0fb8-7693-4b83-9a7b-283859f7c5da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:20.655386 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f269cd38f734209ba0496cf70469608'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:20.656486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f269cd38f734209ba0496cf70469608'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:20.657321 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f269cd38f734209ba0496cf70469608'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:20.658539 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f0fb8-7693-4b83-9a7b-283859f7c5da service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:20.664971 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:20.665883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:20.666173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:20.667697 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] PATH_INFO: `/v3/projects/076218f41775459b9e0b8663029f3d57` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:20.668089 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=076218f41775459b9e0b8663029f3d57)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:20.670792 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:20.684667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1305/2608] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:20 2022] DELETE /identity/v3/users/7f269cd38f734209ba0496cf70469608 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:20.797552 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076218f41775459b9e0b8663029f3d57', 'request_id': 'req-b5485c02-e8d6-4781-a4d8-3df6fe593799'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:20.804482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076218f41775459b9e0b8663029f3d57', 'request_id': 'req-b5485c02-e8d6-4781-a4d8-3df6fe593799'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:20.812075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076218f41775459b9e0b8663029f3d57', 'request_id': 'req-b5485c02-e8d6-4781-a4d8-3df6fe593799'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:20.913707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] The token cache is being invalidate because project 076218f41775459b9e0b8663029f3d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:20.915444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:20.915444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:20.916885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1304/2609] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:20 2022] DELETE /identity/v3/projects/076218f41775459b9e0b8663029f3d57 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:20.920989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:20.922511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:20.973002 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:21.000507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:21.000690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f197c-9a3f-4ac4-8597-865026b4962e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:21.011225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:21.060811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:21.062855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:21.063142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:21.063542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] PATH_INFO: `/v3/users/9ac82e6e46d24fec939d47f181cbf3e3` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:21.065046 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac82e6e46d24fec939d47f181cbf3e3)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:21.067104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:21.074492 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5485c02-e8d6-4781-a4d8-3df6fe593799 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:21.080243 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:21.081419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:21.081710 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:21.081978 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] PATH_INFO: `/v3/projects/d45136a1cdbf4e93b9cf71d33443220f` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:21.083555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45136a1cdbf4e93b9cf71d33443220f)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:21.086406 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:21.256289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:21.271615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac82e6e46d24fec939d47f181cbf3e3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.271813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45136a1cdbf4e93b9cf71d33443220f', 'request_id': 'req-37165576-6f56-4693-8e2f-5c81a1232f0d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.279459 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45136a1cdbf4e93b9cf71d33443220f', 'request_id': 'req-37165576-6f56-4693-8e2f-5c81a1232f0d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.280410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac82e6e46d24fec939d47f181cbf3e3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.285949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45136a1cdbf4e93b9cf71d33443220f', 'request_id': 'req-37165576-6f56-4693-8e2f-5c81a1232f0d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.369986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac82e6e46d24fec939d47f181cbf3e3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.370645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac82e6e46d24fec939d47f181cbf3e3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.371257 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-16131c98-7271-4f78-9280-4fff76c0702d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac82e6e46d24fec939d47f181cbf3e3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.389578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1306/2610] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:20 2022] DELETE /identity/v3/users/9ac82e6e46d24fec939d47f181cbf3e3 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:21.471970 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] The token cache is being invalidate because project d45136a1cdbf4e93b9cf71d33443220f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:21.472263 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.473021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-37165576-6f56-4693-8e2f-5c81a1232f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.474650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1305/2611] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:20 2022] DELETE /identity/v3/projects/d45136a1cdbf4e93b9cf71d33443220f => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:21.480537 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956709ad-ce66-4dd9-8f3c-3798251e8a8d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:21.480878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956709ad-ce66-4dd9-8f3c-3798251e8a8d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:21.481162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956709ad-ce66-4dd9-8f3c-3798251e8a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:21.499910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-956709ad-ce66-4dd9-8f3c-3798251e8a8d None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:21.501115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-956709ad-ce66-4dd9-8f3c-3798251e8a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:21.566132 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1307/2612] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:21.584242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bef69d-5c77-4221-a728-b01ccad1cf73 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:21.584640 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bef69d-5c77-4221-a728-b01ccad1cf73 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:21.584925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bef69d-5c77-4221-a728-b01ccad1cf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:21.607967 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-98bef69d-5c77-4221-a728-b01ccad1cf73 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:21.609266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-98bef69d-5c77-4221-a728-b01ccad1cf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:21.617926 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1306/2613] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:21.621653 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956709ad-ce66-4dd9-8f3c-3798251e8a8d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:21.632029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956709ad-ce66-4dd9-8f3c-3798251e8a8d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:21.637249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b9259-3288-4406-9ae3-0150c7bf982e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:21.637920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b9259-3288-4406-9ae3-0150c7bf982e admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:21.638205 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b9259-3288-4406-9ae3-0150c7bf982e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:21.638542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b9259-3288-4406-9ae3-0150c7bf982e admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:21.639339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b9259-3288-4406-9ae3-0150c7bf982e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:21.639708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b9259-3288-4406-9ae3-0150c7bf982e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:21.641334 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b9259-3288-4406-9ae3-0150c7bf982e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:21.654040 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1308/2614] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:21.658502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bef69d-5c77-4221-a728-b01ccad1cf73 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:21.668393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bef69d-5c77-4221-a728-b01ccad1cf73 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:21.673590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa193f-ab46-435d-bcea-9804504fe113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:21.674320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa193f-ab46-435d-bcea-9804504fe113 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:21.674711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa193f-ab46-435d-bcea-9804504fe113 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:21.674988 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa193f-ab46-435d-bcea-9804504fe113 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:21.675812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa193f-ab46-435d-bcea-9804504fe113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:21.677905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa193f-ab46-435d-bcea-9804504fe113 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:21.754030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1307/2615] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:21 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:21.758252 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b9259-3288-4406-9ae3-0150c7bf982e admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:21.768759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b9259-3288-4406-9ae3-0150c7bf982e admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:21.823986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f98872-dc08-437c-ab1f-29f9553a06f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:21.824578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f98872-dc08-437c-ab1f-29f9553a06f9 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:21.824866 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f98872-dc08-437c-ab1f-29f9553a06f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:21.825147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f98872-dc08-437c-ab1f-29f9553a06f9 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:21.825846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f98872-dc08-437c-ab1f-29f9553a06f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:21.827799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f98872-dc08-437c-ab1f-29f9553a06f9 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:21.840158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa193f-ab46-435d-bcea-9804504fe113 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:21.841936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a5fa193f-ab46-435d-bcea-9804504fe113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:21.857065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa193f-ab46-435d-bcea-9804504fe113 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:21.862248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:21.863745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:21.864218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:21.864501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] PATH_INFO: `/v3/projects/fb752141ce8e491799d9f939a83db53c` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:21.865991 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb752141ce8e491799d9f939a83db53c)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:21.868420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:21.953707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1309/2616] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:21 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:21.957759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f98872-dc08-437c-ab1f-29f9553a06f9 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:21.966877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb752141ce8e491799d9f939a83db53c', 'request_id': 'req-f824698c-0f27-4cd0-bd4b-103a43a30281'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.968114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f98872-dc08-437c-ab1f-29f9553a06f9 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:21.975014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb752141ce8e491799d9f939a83db53c', 'request_id': 'req-f824698c-0f27-4cd0-bd4b-103a43a30281'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.981611 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb752141ce8e491799d9f939a83db53c', 'request_id': 'req-f824698c-0f27-4cd0-bd4b-103a43a30281'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:21.993157 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:22.023970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d0e7f-c9eb-40d3-92d6-e50b3d5e6559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.025150 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d0e7f-c9eb-40d3-92d6-e50b3d5e6559 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.025409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d0e7f-c9eb-40d3-92d6-e50b3d5e6559 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.025682 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d0e7f-c9eb-40d3-92d6-e50b3d5e6559 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.026562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d0e7f-c9eb-40d3-92d6-e50b3d5e6559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.028198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d0e7f-c9eb-40d3-92d6-e50b3d5e6559 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.037913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1310/2617] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:22.042502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d0e7f-c9eb-40d3-92d6-e50b3d5e6559 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.052336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d0e7f-c9eb-40d3-92d6-e50b3d5e6559 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:22.056807 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f98aa3-0f0a-4b4b-b28a-63fef9657c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.057417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f98aa3-0f0a-4b4b-b28a-63fef9657c07 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.057698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f98aa3-0f0a-4b4b-b28a-63fef9657c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.057970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f98aa3-0f0a-4b4b-b28a-63fef9657c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.058833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f98aa3-0f0a-4b4b-b28a-63fef9657c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.060495 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f98aa3-0f0a-4b4b-b28a-63fef9657c07 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.061483 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:22.066482 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] The token cache is being invalidate because project fb752141ce8e491799d9f939a83db53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:22.066884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:22.067565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f824698c-0f27-4cd0-bd4b-103a43a30281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:22.069131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1308/2618] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:21 2022] DELETE /identity/v3/projects/fb752141ce8e491799d9f939a83db53c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:22.072174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1311/2619] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:22.077900 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74990aff-3541-4dae-8fa5-79464ec390ee None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.078413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74990aff-3541-4dae-8fa5-79464ec390ee None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.078740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74990aff-3541-4dae-8fa5-79464ec390ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.079351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f98aa3-0f0a-4b4b-b28a-63fef9657c07 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.081054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-14f98aa3-0f0a-4b4b-b28a-63fef9657c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:22.094631 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f98aa3-0f0a-4b4b-b28a-63fef9657c07 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:22.097092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-74990aff-3541-4dae-8fa5-79464ec390ee None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:22.098301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-74990aff-3541-4dae-8fa5-79464ec390ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:22.099272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf653f2-c02e-4ad8-9baf-64b59c33f773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.100256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf653f2-c02e-4ad8-9baf-64b59c33f773 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.100535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf653f2-c02e-4ad8-9baf-64b59c33f773 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.100805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf653f2-c02e-4ad8-9baf-64b59c33f773 admin admin] PATH_INFO: `/v3/projects/75e99506e5fc403180a7e1d77334ec05/users/c1563f0b55994301b27e4017a325350b/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.108047 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1309/2620] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:22.116809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf653f2-c02e-4ad8-9baf-64b59c33f773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e99506e5fc403180a7e1d77334ec05, user_id=c1563f0b55994301b27e4017a325350b, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.116973 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d0c9b-b3f6-4f0e-b790-afeb426a10ce None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.118581 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d0c9b-b3f6-4f0e-b790-afeb426a10ce None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.118581 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d0c9b-b3f6-4f0e-b790-afeb426a10ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.119492 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf653f2-c02e-4ad8-9baf-64b59c33f773 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.135646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ba8d0c9b-b3f6-4f0e-b790-afeb426a10ce None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:22.136859 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ba8d0c9b-b3f6-4f0e-b790-afeb426a10ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:22.144905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1310/2621] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:22.148949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d0c9b-b3f6-4f0e-b790-afeb426a10ce None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.158223 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d0c9b-b3f6-4f0e-b790-afeb426a10ce None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:22.162908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1cc27-340a-4ea0-a210-ba4a9ac77bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.163571 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1cc27-340a-4ea0-a210-ba4a9ac77bee admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.163857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1cc27-340a-4ea0-a210-ba4a9ac77bee admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.164143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1cc27-340a-4ea0-a210-ba4a9ac77bee admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.165417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1cc27-340a-4ea0-a210-ba4a9ac77bee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:22.165732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1cc27-340a-4ea0-a210-ba4a9ac77bee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.167348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1cc27-340a-4ea0-a210-ba4a9ac77bee admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.175716 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1312/2622] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:22 2022] PUT /identity/v3/projects/75e99506e5fc403180a7e1d77334ec05/users/c1563f0b55994301b27e4017a325350b/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:22.179869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1311/2623] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:06:22.183817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf653f2-c02e-4ad8-9baf-64b59c33f773 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.187344 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c587-d38b-4478-ab4a-219b53bb1a66 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.187795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c587-d38b-4478-ab4a-219b53bb1a66 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.188151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c587-d38b-4478-ab4a-219b53bb1a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.214146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-bdb9c587-d38b-4478-ab4a-219b53bb1a66 None None] MFA Rules not processed for user `c1563f0b55994301b27e4017a325350b`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:22.215395 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bdb9c587-d38b-4478-ab4a-219b53bb1a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:22.248026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf653f2-c02e-4ad8-9baf-64b59c33f773 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:22.252701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da8f3-d989-410f-b7f6-48b198674203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.253362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da8f3-d989-410f-b7f6-48b198674203 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.253645 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da8f3-d989-410f-b7f6-48b198674203 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.253920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da8f3-d989-410f-b7f6-48b198674203 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.254660 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da8f3-d989-410f-b7f6-48b198674203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.256567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da8f3-d989-410f-b7f6-48b198674203 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.281143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1312/2624] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:22.292771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9c587-d38b-4478-ab4a-219b53bb1a66 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.294374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bdb9c587-d38b-4478-ab4a-219b53bb1a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:22.319539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1313/2625] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:22 2022] POST /identity/v3/projects => generated 401 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:22.323655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da8f3-d989-410f-b7f6-48b198674203 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.367566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9c587-d38b-4478-ab4a-219b53bb1a66 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:22.386287 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da8f3-d989-410f-b7f6-48b198674203 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:22.394344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba91a0-a830-4eb2-8a5b-f7bc97543492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.395037 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba91a0-a830-4eb2-8a5b-f7bc97543492 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.395343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba91a0-a830-4eb2-8a5b-f7bc97543492 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.395614 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba91a0-a830-4eb2-8a5b-f7bc97543492 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.396295 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba91a0-a830-4eb2-8a5b-f7bc97543492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.399202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba91a0-a830-4eb2-8a5b-f7bc97543492 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.420439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:22.431771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360c0ec-866d-4427-a61e-2d4a4119e895 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.432542 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360c0ec-866d-4427-a61e-2d4a4119e895 service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.432831 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360c0ec-866d-4427-a61e-2d4a4119e895 service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.433106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360c0ec-866d-4427-a61e-2d4a4119e895 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.475730 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1314/2626] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:22 2022] POST /identity/v3/users => generated 301 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:22.480775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba91a0-a830-4eb2-8a5b-f7bc97543492 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.485829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:22.490605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360c0ec-866d-4427-a61e-2d4a4119e895 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.490966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba91a0-a830-4eb2-8a5b-f7bc97543492 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:22.492202 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360c0ec-866d-4427-a61e-2d4a4119e895 service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.501922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c271c-54d9-472d-adb9-1d5466af879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.502123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1313/2627] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:22.502619 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c271c-54d9-472d-adb9-1d5466af879a admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.502901 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c271c-54d9-472d-adb9-1d5466af879a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.503173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c271c-54d9-472d-adb9-1d5466af879a admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.503898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c271c-54d9-472d-adb9-1d5466af879a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.505612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c271c-54d9-472d-adb9-1d5466af879a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.520562 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1315/2628] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:22.521633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360c0ec-866d-4427-a61e-2d4a4119e895 service nova] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.532201 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360c0ec-866d-4427-a61e-2d4a4119e895 service nova] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:22.536786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8a85d-645d-4a3e-bd6e-1b975d5a8927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.537449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8a85d-645d-4a3e-bd6e-1b975d5a8927 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.537737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8a85d-645d-4a3e-bd6e-1b975d5a8927 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.538019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8a85d-645d-4a3e-bd6e-1b975d5a8927 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.538782 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8a85d-645d-4a3e-bd6e-1b975d5a8927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.540915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8a85d-645d-4a3e-bd6e-1b975d5a8927 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.551160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1314/2629] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:22.555515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c271c-54d9-472d-adb9-1d5466af879a admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.565646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c271c-54d9-472d-adb9-1d5466af879a admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:22.570351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f5004-b7fb-41de-9719-3031b3d2e502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.571307 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f5004-b7fb-41de-9719-3031b3d2e502 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.571589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f5004-b7fb-41de-9719-3031b3d2e502 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.571868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f5004-b7fb-41de-9719-3031b3d2e502 admin admin] PATH_INFO: `/v3/projects/8d9e75859a054f59ae8fd1902ab04459/users/70124f633ab1454eaaa823a3749ed65a/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.590446 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f5004-b7fb-41de-9719-3031b3d2e502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9e75859a054f59ae8fd1902ab04459, user_id=70124f633ab1454eaaa823a3749ed65a, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.593174 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f5004-b7fb-41de-9719-3031b3d2e502 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.661502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1316/2630] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:22 2022] PUT /identity/v3/projects/8d9e75859a054f59ae8fd1902ab04459/users/70124f633ab1454eaaa823a3749ed65a/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:22.672567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a12936-703c-47a8-a00e-b43788cb598d None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.673058 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a12936-703c-47a8-a00e-b43788cb598d None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.673182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a12936-703c-47a8-a00e-b43788cb598d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.700170 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c3a12936-703c-47a8-a00e-b43788cb598d None None] MFA Rules not processed for user `70124f633ab1454eaaa823a3749ed65a`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:22.701384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c3a12936-703c-47a8-a00e-b43788cb598d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:22.765968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1315/2631] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:22.770521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f5004-b7fb-41de-9719-3031b3d2e502 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.841656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f5004-b7fb-41de-9719-3031b3d2e502 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:22.848526 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c194d-3ede-4b9d-ad73-b29cd31b440c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:22.849193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c194d-3ede-4b9d-ad73-b29cd31b440c admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:22.849461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c194d-3ede-4b9d-ad73-b29cd31b440c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:22.849734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c194d-3ede-4b9d-ad73-b29cd31b440c admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:22.850416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c194d-3ede-4b9d-ad73-b29cd31b440c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:22.853000 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c194d-3ede-4b9d-ad73-b29cd31b440c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:22.934435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1317/2632] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:22 2022] POST /identity/v3/projects => generated 401 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:22.935631 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a12936-703c-47a8-a00e-b43788cb598d None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:22.946444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a12936-703c-47a8-a00e-b43788cb598d None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:23.003368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d1136-7798-4c91-9465-0abe3521e378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:23.004040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d1136-7798-4c91-9465-0abe3521e378 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:23.004321 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d1136-7798-4c91-9465-0abe3521e378 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:23.004614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d1136-7798-4c91-9465-0abe3521e378 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:23.005266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d1136-7798-4c91-9465-0abe3521e378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:23.007234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d1136-7798-4c91-9465-0abe3521e378 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:23.107096 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1316/2633] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:22 2022] POST /identity/v3/users => generated 307 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:23.110987 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c194d-3ede-4b9d-ad73-b29cd31b440c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:23.125261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c194d-3ede-4b9d-ad73-b29cd31b440c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:23.130221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db63c7-addb-4eb8-992b-5b9896295fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:23.130924 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db63c7-addb-4eb8-992b-5b9896295fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:23.131212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db63c7-addb-4eb8-992b-5b9896295fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:23.131494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db63c7-addb-4eb8-992b-5b9896295fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:23.132231 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db63c7-addb-4eb8-992b-5b9896295fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:23.134497 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db63c7-addb-4eb8-992b-5b9896295fc3 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:23.145261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1318/2634] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:23.152208 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d1136-7798-4c91-9465-0abe3521e378 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:23.160500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:23.165358 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d1136-7798-4c91-9465-0abe3521e378 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:23.170662 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d175a-96d7-4fce-87db-c21ba4bad0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:23.171565 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d175a-96d7-4fce-87db-c21ba4bad0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:23.171849 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d175a-96d7-4fce-87db-c21ba4bad0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:23.172129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d175a-96d7-4fce-87db-c21ba4bad0cb admin admin] PATH_INFO: `/v3/projects/1c9671c64c4d421b89de458ef58b82f3/users/974c60c9a91d4624adad7bd7efb9795f/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:23.189912 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d175a-96d7-4fce-87db-c21ba4bad0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9671c64c4d421b89de458ef58b82f3, user_id=974c60c9a91d4624adad7bd7efb9795f, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:23.192777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d175a-96d7-4fce-87db-c21ba4bad0cb admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:23.273067 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1317/2635] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:23 2022] PUT /identity/v3/projects/1c9671c64c4d421b89de458ef58b82f3/users/974c60c9a91d4624adad7bd7efb9795f/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:23.277603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db63c7-addb-4eb8-992b-5b9896295fc3 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:23.287955 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db63c7-addb-4eb8-992b-5b9896295fc3 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:23.342889 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700569a-51f9-46fd-a501-c54fdb3c07f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:23.343550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700569a-51f9-46fd-a501-c54fdb3c07f1 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:23.343833 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700569a-51f9-46fd-a501-c54fdb3c07f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:23.344105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700569a-51f9-46fd-a501-c54fdb3c07f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:23.344822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700569a-51f9-46fd-a501-c54fdb3c07f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:23.346420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700569a-51f9-46fd-a501-c54fdb3c07f1 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:23.356168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1319/2636] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:23.360462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d175a-96d7-4fce-87db-c21ba4bad0cb admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:23.371352 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d175a-96d7-4fce-87db-c21ba4bad0cb admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:23.376175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ca70c-3440-4035-9df3-d6c92d1e709b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:23.377077 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ca70c-3440-4035-9df3-d6c92d1e709b admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:23.377341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ca70c-3440-4035-9df3-d6c92d1e709b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:23.377618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ca70c-3440-4035-9df3-d6c92d1e709b admin admin] PATH_INFO: `/v3/projects/1c9671c64c4d421b89de458ef58b82f3/users/974c60c9a91d4624adad7bd7efb9795f/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:23.379647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ca70c-3440-4035-9df3-d6c92d1e709b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9671c64c4d421b89de458ef58b82f3, user_id=974c60c9a91d4624adad7bd7efb9795f, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:23.382486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ca70c-3440-4035-9df3-d6c92d1e709b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:23.393153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1318/2637] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:23 2022] PUT /identity/v3/projects/1c9671c64c4d421b89de458ef58b82f3/users/974c60c9a91d4624adad7bd7efb9795f/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:23.403970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4e621-16c9-4924-9c6a-47033c7adc42 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:23.404286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4e621-16c9-4924-9c6a-47033c7adc42 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:23.404567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4e621-16c9-4924-9c6a-47033c7adc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:23.430202 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-53b4e621-16c9-4924-9c6a-47033c7adc42 None None] MFA Rules not processed for user `974c60c9a91d4624adad7bd7efb9795f`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:23.431886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-53b4e621-16c9-4924-9c6a-47033c7adc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:23.457426 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:23.499852 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1320/2638] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:23 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:23.512197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ca70c-3440-4035-9df3-d6c92d1e709b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:23.514222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d99ca70c-3440-4035-9df3-d6c92d1e709b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:23.573879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ca70c-3440-4035-9df3-d6c92d1e709b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:23.577631 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6ab3b-d69d-4dac-a7dc-c8175b37e446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:23.585097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6ab3b-d69d-4dac-a7dc-c8175b37e446 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:23.585097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6ab3b-d69d-4dac-a7dc-c8175b37e446 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:23.585097 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6ab3b-d69d-4dac-a7dc-c8175b37e446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:23.587837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6ab3b-d69d-4dac-a7dc-c8175b37e446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:23.589256 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6ab3b-d69d-4dac-a7dc-c8175b37e446 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:23.598789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1319/2639] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:23 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:23.851229 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4e621-16c9-4924-9c6a-47033c7adc42 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:23.853534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-53b4e621-16c9-4924-9c6a-47033c7adc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:23.865840 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4e621-16c9-4924-9c6a-47033c7adc42 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:23.870209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee416b-dc24-42e0-bb7a-555a84a3e129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:23.871034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee416b-dc24-42e0-bb7a-555a84a3e129 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:23.871317 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee416b-dc24-42e0-bb7a-555a84a3e129 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:23.871593 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee416b-dc24-42e0-bb7a-555a84a3e129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:23.934954 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee416b-dc24-42e0-bb7a-555a84a3e129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:23.936406 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee416b-dc24-42e0-bb7a-555a84a3e129 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:23.945431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1321/2640] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:25.274925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6ab3b-d69d-4dac-a7dc-c8175b37e446 service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:25.276498 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f8a6ab3b-d69d-4dac-a7dc-c8175b37e446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:25.342167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6ab3b-d69d-4dac-a7dc-c8175b37e446 service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:25.346839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:25.349092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:25.349092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:25.349092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] PATH_INFO: `/v3/users/859f6afc5ac74a9aaebaf73b72dcccc1` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:25.349526 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=859f6afc5ac74a9aaebaf73b72dcccc1)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:25.351480 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:25.560020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '859f6afc5ac74a9aaebaf73b72dcccc1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:25.570171 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '859f6afc5ac74a9aaebaf73b72dcccc1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:25.571374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '859f6afc5ac74a9aaebaf73b72dcccc1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:25.655399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '859f6afc5ac74a9aaebaf73b72dcccc1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:25.656092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-ce633029-a011-46da-891f-8354c3d58f99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '859f6afc5ac74a9aaebaf73b72dcccc1'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:25.665603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:25.702867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1320/2641] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:25 2022] DELETE /identity/v3/users/859f6afc5ac74a9aaebaf73b72dcccc1 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:25.703383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee416b-dc24-42e0-bb7a-555a84a3e129 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:25.703832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751ecce-eac2-4678-91cb-224e59ea38fd None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:25.703832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751ecce-eac2-4678-91cb-224e59ea38fd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:25.703832 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751ecce-eac2-4678-91cb-224e59ea38fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:25.706941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee416b-dc24-42e0-bb7a-555a84a3e129 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:25.718829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c751ecce-eac2-4678-91cb-224e59ea38fd None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:25.720249 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c751ecce-eac2-4678-91cb-224e59ea38fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:25.777414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:25.778285 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:25.778603 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:25.778907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] PATH_INFO: `/v3/users/c1563f0b55994301b27e4017a325350b` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:25.782128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1563f0b55994301b27e4017a325350b)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:25.784086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:25.810496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1321/2642] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:25.823043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751ecce-eac2-4678-91cb-224e59ea38fd None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:25.824568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c751ecce-eac2-4678-91cb-224e59ea38fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:25.882890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751ecce-eac2-4678-91cb-224e59ea38fd None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:25.887989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf362e7-626d-4ae0-927a-ec46ad2aaee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:25.887989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf362e7-626d-4ae0-927a-ec46ad2aaee2 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:25.887989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf362e7-626d-4ae0-927a-ec46ad2aaee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:25.887989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf362e7-626d-4ae0-927a-ec46ad2aaee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:25.904520 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf362e7-626d-4ae0-927a-ec46ad2aaee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:25.906113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf362e7-626d-4ae0-927a-ec46ad2aaee2 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:25.918333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1322/2643] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:26.001149 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1563f0b55994301b27e4017a325350b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.011555 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1563f0b55994301b27e4017a325350b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.094799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1563f0b55994301b27e4017a325350b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.095409 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1563f0b55994301b27e4017a325350b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.096065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1563f0b55994301b27e4017a325350b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.097689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:26.117624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1322/2644] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:25 2022] DELETE /identity/v3/users/c1563f0b55994301b27e4017a325350b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:26.122279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e9e41-7c2b-48ab-aae4-3a2f6770c313 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.122850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e9e41-7c2b-48ab-aae4-3a2f6770c313 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.123245 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e9e41-7c2b-48ab-aae4-3a2f6770c313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.142735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3d8e9e41-7c2b-48ab-aae4-3a2f6770c313 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:26.142735 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3d8e9e41-7c2b-48ab-aae4-3a2f6770c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:26.210644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1323/2645] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:26.220910 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:26.223457 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:26.284587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7904a4d-c718-4f14-b06d-529adfee6c61 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:26.287099 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2042c73-4765-4825-8395-ae8502c6ff00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:26.287759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2042c73-4765-4825-8395-ae8502c6ff00 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.288029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2042c73-4765-4825-8395-ae8502c6ff00 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.288304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2042c73-4765-4825-8395-ae8502c6ff00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.298577 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2042c73-4765-4825-8395-ae8502c6ff00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:26.300802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2042c73-4765-4825-8395-ae8502c6ff00 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:26.311821 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1323/2646] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:26.364368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e9e41-7c2b-48ab-aae4-3a2f6770c313 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:26.365902 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3d8e9e41-7c2b-48ab-aae4-3a2f6770c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:26.380473 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e9e41-7c2b-48ab-aae4-3a2f6770c313 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:26.385982 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:26.396107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.396107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.396107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] PATH_INFO: `/v3/projects/ce1e92a4ddad48a0a2156db83def1660` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.396107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce1e92a4ddad48a0a2156db83def1660)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:26.396107 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:26.520711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:26.536081 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce1e92a4ddad48a0a2156db83def1660', 'request_id': 'req-a9a487a7-fca1-486b-a7dd-65810ff91d4c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.542236 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce1e92a4ddad48a0a2156db83def1660', 'request_id': 'req-a9a487a7-fca1-486b-a7dd-65810ff91d4c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.548397 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce1e92a4ddad48a0a2156db83def1660', 'request_id': 'req-a9a487a7-fca1-486b-a7dd-65810ff91d4c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.626654 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] The token cache is being invalidate because project ce1e92a4ddad48a0a2156db83def1660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:26.626965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.627594 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a9a487a7-fca1-486b-a7dd-65810ff91d4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.629537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1324/2647] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:26 2022] DELETE /identity/v3/projects/ce1e92a4ddad48a0a2156db83def1660 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:26.635256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b47cf2-6f7a-41e3-bb14-ac7515b1372a None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.635571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b47cf2-6f7a-41e3-bb14-ac7515b1372a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.635844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b47cf2-6f7a-41e3-bb14-ac7515b1372a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.654269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-57b47cf2-6f7a-41e3-bb14-ac7515b1372a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:26.657942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-57b47cf2-6f7a-41e3-bb14-ac7515b1372a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:26.715697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1324/2648] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:26.723918 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225defa6-7e06-4573-8a3b-57cb83d40747 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.724208 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225defa6-7e06-4573-8a3b-57cb83d40747 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.724488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225defa6-7e06-4573-8a3b-57cb83d40747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.742623 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-225defa6-7e06-4573-8a3b-57cb83d40747 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:26.744351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-225defa6-7e06-4573-8a3b-57cb83d40747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:26.752303 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1325/2649] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:26.756580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b47cf2-6f7a-41e3-bb14-ac7515b1372a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:26.766026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225defa6-7e06-4573-8a3b-57cb83d40747 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:26.767600 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-225defa6-7e06-4573-8a3b-57cb83d40747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:26.769522 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b47cf2-6f7a-41e3-bb14-ac7515b1372a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:26.777691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009bd09-01b8-4bdf-8fdb-a337dbdc963b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:26.777691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009bd09-01b8-4bdf-8fdb-a337dbdc963b admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.777691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009bd09-01b8-4bdf-8fdb-a337dbdc963b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.781768 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225defa6-7e06-4573-8a3b-57cb83d40747 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:26.783604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009bd09-01b8-4bdf-8fdb-a337dbdc963b admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.784498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009bd09-01b8-4bdf-8fdb-a337dbdc963b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:26.784830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009bd09-01b8-4bdf-8fdb-a337dbdc963b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:26.786479 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009bd09-01b8-4bdf-8fdb-a337dbdc963b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:26.786815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:26.787674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.788082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.788360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] PATH_INFO: `/v3/projects/75e99506e5fc403180a7e1d77334ec05` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.790411 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e99506e5fc403180a7e1d77334ec05)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:26.792413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:26.803766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1325/2650] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:26.808903 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009bd09-01b8-4bdf-8fdb-a337dbdc963b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:26.818971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009bd09-01b8-4bdf-8fdb-a337dbdc963b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:26.823706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f94f8-1a62-4926-ae69-65ac31f0c723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:26.824366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f94f8-1a62-4926-ae69-65ac31f0c723 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.824646 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f94f8-1a62-4926-ae69-65ac31f0c723 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.824919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f94f8-1a62-4926-ae69-65ac31f0c723 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.825648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f94f8-1a62-4926-ae69-65ac31f0c723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:26.827565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f94f8-1a62-4926-ae69-65ac31f0c723 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:26.925158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e99506e5fc403180a7e1d77334ec05', 'request_id': 'req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.932109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e99506e5fc403180a7e1d77334ec05', 'request_id': 'req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.938048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e99506e5fc403180a7e1d77334ec05', 'request_id': 'req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.942932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1326/2651] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:26 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:26.947104 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f94f8-1a62-4926-ae69-65ac31f0c723 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:26.996124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:27.009335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f94f8-1a62-4926-ae69-65ac31f0c723 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.020661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef292a1b-c6d8-430a-8f49-5c9f12a69a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.020661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef292a1b-c6d8-430a-8f49-5c9f12a69a27 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.020661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef292a1b-c6d8-430a-8f49-5c9f12a69a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.020661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef292a1b-c6d8-430a-8f49-5c9f12a69a27 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.021020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] The token cache is being invalidate because project 75e99506e5fc403180a7e1d77334ec05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:27.021135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef292a1b-c6d8-430a-8f49-5c9f12a69a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.021225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.021225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-f8f16d48-8926-4ea9-9bc9-3e4dc78d13a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.021398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef292a1b-c6d8-430a-8f49-5c9f12a69a27 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.021487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1326/2652] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:26 2022] DELETE /identity/v3/projects/75e99506e5fc403180a7e1d77334ec05 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:27.026591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59851-e11a-4478-ba90-cd707d071ddd None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.026885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59851-e11a-4478-ba90-cd707d071ddd None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.027167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59851-e11a-4478-ba90-cd707d071ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.050711 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-f8c59851-e11a-4478-ba90-cd707d071ddd None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:27.051962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f8c59851-e11a-4478-ba90-cd707d071ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.060624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1327/2653] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:27.068283 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54ec50-a717-471b-b2c8-695e25bc6a10 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.068572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54ec50-a717-471b-b2c8-695e25bc6a10 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.068875 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54ec50-a717-471b-b2c8-695e25bc6a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.086857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ca54ec50-a717-471b-b2c8-695e25bc6a10 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:27.088070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ca54ec50-a717-471b-b2c8-695e25bc6a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.099378 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1328/2654] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:27.100572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54ec50-a717-471b-b2c8-695e25bc6a10 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.105654 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1327/2655] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:26 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:27.106744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef292a1b-c6d8-430a-8f49-5c9f12a69a27 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.108533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ef292a1b-c6d8-430a-8f49-5c9f12a69a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.112188 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54ec50-a717-471b-b2c8-695e25bc6a10 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.116856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f32a9d-20d0-4f50-89e1-a47f14f1dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.117637 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f32a9d-20d0-4f50-89e1-a47f14f1dc96 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.118539 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f32a9d-20d0-4f50-89e1-a47f14f1dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.118813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f32a9d-20d0-4f50-89e1-a47f14f1dc96 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.119546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f32a9d-20d0-4f50-89e1-a47f14f1dc96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:27.119847 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f32a9d-20d0-4f50-89e1-a47f14f1dc96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.121438 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f32a9d-20d0-4f50-89e1-a47f14f1dc96 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.123561 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef292a1b-c6d8-430a-8f49-5c9f12a69a27 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.131630 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8594bd-12ab-497e-bb7f-5871c6b5ffac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.132344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8594bd-12ab-497e-bb7f-5871c6b5ffac admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.132628 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8594bd-12ab-497e-bb7f-5871c6b5ffac admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.132942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8594bd-12ab-497e-bb7f-5871c6b5ffac admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.133685 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8594bd-12ab-497e-bb7f-5871c6b5ffac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.135363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8594bd-12ab-497e-bb7f-5871c6b5ffac admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.137175 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1329/2656] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:27.141272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f32a9d-20d0-4f50-89e1-a47f14f1dc96 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.148480 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1328/2657] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:27.149620 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8594bd-12ab-497e-bb7f-5871c6b5ffac admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.156324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f32a9d-20d0-4f50-89e1-a47f14f1dc96 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.159359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8594bd-12ab-497e-bb7f-5871c6b5ffac admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.161671 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d9e12-6ddb-47a4-b54a-17f8aa1e395c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.162272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9e12-6ddb-47a4-b54a-17f8aa1e395c admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.162675 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9e12-6ddb-47a4-b54a-17f8aa1e395c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.162944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9e12-6ddb-47a4-b54a-17f8aa1e395c admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.163604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d9e12-6ddb-47a4-b54a-17f8aa1e395c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.165890 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73e6d2-d295-44bf-a01a-506673b52457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.166106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d9e12-6ddb-47a4-b54a-17f8aa1e395c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.166581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e6d2-d295-44bf-a01a-506673b52457 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.166864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e6d2-d295-44bf-a01a-506673b52457 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.167136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e6d2-d295-44bf-a01a-506673b52457 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.168026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73e6d2-d295-44bf-a01a-506673b52457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.171080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73e6d2-d295-44bf-a01a-506673b52457 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.184267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1329/2658] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:27.189346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73e6d2-d295-44bf-a01a-506673b52457 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.199452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73e6d2-d295-44bf-a01a-506673b52457 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.204170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2342f40-5e03-4ae9-a4dd-d128465044f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.205055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2342f40-5e03-4ae9-a4dd-d128465044f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.205335 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2342f40-5e03-4ae9-a4dd-d128465044f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.205609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2342f40-5e03-4ae9-a4dd-d128465044f7 admin admin] PATH_INFO: `/v3/projects/a39f82493df2479395de88147e2f8292/users/a01a4b9868714295a1000e0a5a81143b/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.221178 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2342f40-5e03-4ae9-a4dd-d128465044f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39f82493df2479395de88147e2f8292, user_id=a01a4b9868714295a1000e0a5a81143b, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.223882 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2342f40-5e03-4ae9-a4dd-d128465044f7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.247610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1330/2659] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:27 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:27.249734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d9e12-6ddb-47a4-b54a-17f8aa1e395c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.296551 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:27.307850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1330/2660] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:27 2022] PUT /identity/v3/projects/a39f82493df2479395de88147e2f8292/users/a01a4b9868714295a1000e0a5a81143b/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:27.313239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad04a0-d12c-4004-9fa5-159040e317d4 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.313700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad04a0-d12c-4004-9fa5-159040e317d4 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.314129 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad04a0-d12c-4004-9fa5-159040e317d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.315000 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d9e12-6ddb-47a4-b54a-17f8aa1e395c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.333719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-76ad04a0-d12c-4004-9fa5-159040e317d4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:27.334984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-76ad04a0-d12c-4004-9fa5-159040e317d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.382974 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7d670-298f-47e2-9a20-7a93cb3e7f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.382974 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7d670-298f-47e2-9a20-7a93cb3e7f7b admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.383731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7d670-298f-47e2-9a20-7a93cb3e7f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.383731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7d670-298f-47e2-9a20-7a93cb3e7f7b admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.384009 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7d670-298f-47e2-9a20-7a93cb3e7f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.385928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7d670-298f-47e2-9a20-7a93cb3e7f7b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.421279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1331/2661] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:27.432674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad04a0-d12c-4004-9fa5-159040e317d4 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.434699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-76ad04a0-d12c-4004-9fa5-159040e317d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.485322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1331/2662] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:27 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:27.489330 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7d670-298f-47e2-9a20-7a93cb3e7f7b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.499707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7d670-298f-47e2-9a20-7a93cb3e7f7b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.500025 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad04a0-d12c-4004-9fa5-159040e317d4 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.503976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006abb50-4564-4c7f-b608-59558d1268b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.504635 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006abb50-4564-4c7f-b608-59558d1268b6 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.504914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006abb50-4564-4c7f-b608-59558d1268b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.505195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006abb50-4564-4c7f-b608-59558d1268b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.511286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba418df-6217-4561-a400-ebc52ef9ca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.511968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba418df-6217-4561-a400-ebc52ef9ca8e admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.515509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba418df-6217-4561-a400-ebc52ef9ca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.515509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba418df-6217-4561-a400-ebc52ef9ca8e admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.515509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba418df-6217-4561-a400-ebc52ef9ca8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.515509 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba418df-6217-4561-a400-ebc52ef9ca8e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.516272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006abb50-4564-4c7f-b608-59558d1268b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.517505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006abb50-4564-4c7f-b608-59558d1268b6 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.526377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1332/2663] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:27.530311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba418df-6217-4561-a400-ebc52ef9ca8e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.530894 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1332/2664] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:27.540985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba418df-6217-4561-a400-ebc52ef9ca8e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.546093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b9ba1-9370-4a47-8bc2-e03b9874589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.554501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b9ba1-9370-4a47-8bc2-e03b9874589a admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.554501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b9ba1-9370-4a47-8bc2-e03b9874589a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.554501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b9ba1-9370-4a47-8bc2-e03b9874589a admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.554501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b9ba1-9370-4a47-8bc2-e03b9874589a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.554501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b9ba1-9370-4a47-8bc2-e03b9874589a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.570523 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1333/2665] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:27.574449 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006abb50-4564-4c7f-b608-59558d1268b6 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.585156 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006abb50-4564-4c7f-b608-59558d1268b6 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.590665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be2682-810a-414b-af80-101c9e7842ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.591553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be2682-810a-414b-af80-101c9e7842ff admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.592759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be2682-810a-414b-af80-101c9e7842ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.592759 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be2682-810a-414b-af80-101c9e7842ff admin admin] PATH_INFO: `/v3/projects/909441cb99d247109c47fc531020f38c/users/199efed2957d4854af7adc758e60196d/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.608399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be2682-810a-414b-af80-101c9e7842ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=909441cb99d247109c47fc531020f38c, user_id=199efed2957d4854af7adc758e60196d, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.611084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be2682-810a-414b-af80-101c9e7842ff admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.695907 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1333/2666] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:27 2022] PUT /identity/v3/projects/909441cb99d247109c47fc531020f38c/users/199efed2957d4854af7adc758e60196d/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:27.707862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bd3f8-eecb-46cd-8534-b77c37d0c0ea None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.708471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bd3f8-eecb-46cd-8534-b77c37d0c0ea None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.708969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bd3f8-eecb-46cd-8534-b77c37d0c0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.748237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-ea5bd3f8-eecb-46cd-8534-b77c37d0c0ea None None] MFA Rules not processed for user `199efed2957d4854af7adc758e60196d`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:27.749630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ea5bd3f8-eecb-46cd-8534-b77c37d0c0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.819987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1334/2667] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:27.833922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be2682-810a-414b-af80-101c9e7842ff admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.835771 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-c1be2682-810a-414b-af80-101c9e7842ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.850183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:27.912711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be2682-810a-414b-af80-101c9e7842ff admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.918308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfcaf1-38b2-4b57-a039-a3e04478cad7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.919249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfcaf1-38b2-4b57-a039-a3e04478cad7 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.919547 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfcaf1-38b2-4b57-a039-a3e04478cad7 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.919816 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfcaf1-38b2-4b57-a039-a3e04478cad7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.928266 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfcaf1-38b2-4b57-a039-a3e04478cad7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.929949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfcaf1-38b2-4b57-a039-a3e04478cad7 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.940868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1334/2668] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:28.185215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bd3f8-eecb-46cd-8534-b77c37d0c0ea None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:28.238916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:28.252538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bd3f8-eecb-46cd-8534-b77c37d0c0ea None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:28.257082 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:28.257957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.258351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.258628 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] PATH_INFO: `/v3/users/199efed2957d4854af7adc758e60196d` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.259812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=199efed2957d4854af7adc758e60196d)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:28.261779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:28.475292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199efed2957d4854af7adc758e60196d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:28.482200 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199efed2957d4854af7adc758e60196d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:28.482913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199efed2957d4854af7adc758e60196d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:28.545501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199efed2957d4854af7adc758e60196d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:28.546254 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199efed2957d4854af7adc758e60196d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:28.564393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1335/2669] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:28 2022] DELETE /identity/v3/users/199efed2957d4854af7adc758e60196d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:28.568766 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b2782-518d-4c0d-9146-c704ae6a6598 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.569056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b2782-518d-4c0d-9146-c704ae6a6598 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.569331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b2782-518d-4c0d-9146-c704ae6a6598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.587777 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-400b2782-518d-4c0d-9146-c704ae6a6598 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:28.589004 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-400b2782-518d-4c0d-9146-c704ae6a6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:28.650570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1335/2670] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:28.660160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:28.661790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:28.732260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5eceed-1e8b-4644-a8aa-c5cf744c8178 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:28.736021 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10164439-3775-4fab-94bb-127cea29175c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:28.736682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10164439-3775-4fab-94bb-127cea29175c service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.736963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10164439-3775-4fab-94bb-127cea29175c service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.737240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10164439-3775-4fab-94bb-127cea29175c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.747373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10164439-3775-4fab-94bb-127cea29175c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:28.749447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10164439-3775-4fab-94bb-127cea29175c service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:28.758316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1336/2671] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:29.139550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b2782-518d-4c0d-9146-c704ae6a6598 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:29.141173 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-400b2782-518d-4c0d-9146-c704ae6a6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:29.153962 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b2782-518d-4c0d-9146-c704ae6a6598 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:29.159579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:29.160467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.160758 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.161032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] PATH_INFO: `/v3/users/70124f633ab1454eaaa823a3749ed65a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.162602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70124f633ab1454eaaa823a3749ed65a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:29.164545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:29.332567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10164439-3775-4fab-94bb-127cea29175c service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:29.333992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-10164439-3775-4fab-94bb-127cea29175c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:29.348003 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10164439-3775-4fab-94bb-127cea29175c service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:29.353121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:29.354024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.354322 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.354682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] PATH_INFO: `/v3/projects/909441cb99d247109c47fc531020f38c` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.356019 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=909441cb99d247109c47fc531020f38c)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:29.357972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:29.400865 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70124f633ab1454eaaa823a3749ed65a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:29.408557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70124f633ab1454eaaa823a3749ed65a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:29.474431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '909441cb99d247109c47fc531020f38c', 'request_id': 'req-7accf3cb-ee2c-434e-8eef-636a241f39b2'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:29.479435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70124f633ab1454eaaa823a3749ed65a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:29.481693 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70124f633ab1454eaaa823a3749ed65a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:29.483978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-2276ebea-a2d5-4b30-af43-025e829dbd61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70124f633ab1454eaaa823a3749ed65a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:29.484530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '909441cb99d247109c47fc531020f38c', 'request_id': 'req-7accf3cb-ee2c-434e-8eef-636a241f39b2'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:29.498010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '909441cb99d247109c47fc531020f38c', 'request_id': 'req-7accf3cb-ee2c-434e-8eef-636a241f39b2'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:29.502535 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:29.510052 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1336/2672] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:29 2022] DELETE /identity/v3/users/70124f633ab1454eaaa823a3749ed65a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:29.514882 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebd5e0-1c58-4b40-99b8-0f73572f69db None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.515246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebd5e0-1c58-4b40-99b8-0f73572f69db None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.515602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebd5e0-1c58-4b40-99b8-0f73572f69db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.537228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-76ebd5e0-1c58-4b40-99b8-0f73572f69db None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:29.547748 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-76ebd5e0-1c58-4b40-99b8-0f73572f69db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:29.550323 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:29.601244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] The token cache is being invalidate because project 909441cb99d247109c47fc531020f38c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:29.601756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:29.604272 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-7accf3cb-ee2c-434e-8eef-636a241f39b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:29.607255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1337/2673] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:29 2022] DELETE /identity/v3/projects/909441cb99d247109c47fc531020f38c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:29.612063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b62c0-9856-43eb-b359-ad2508d073d4 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.615800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b62c0-9856-43eb-b359-ad2508d073d4 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.615800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b62c0-9856-43eb-b359-ad2508d073d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.619658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1337/2674] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:29.641709 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebd5e0-1c58-4b40-99b8-0f73572f69db None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:29.643480 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-76ebd5e0-1c58-4b40-99b8-0f73572f69db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:29.654658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-034b62c0-9856-43eb-b359-ad2508d073d4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:29.656913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-034b62c0-9856-43eb-b359-ad2508d073d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:29.670763 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1338/2675] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:29.680178 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5934-c64f-4e62-b633-c760746c1a37 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.681354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5934-c64f-4e62-b633-c760746c1a37 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.681663 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be5934-c64f-4e62-b633-c760746c1a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.708721 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-17be5934-c64f-4e62-b633-c760746c1a37 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:29.710113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-17be5934-c64f-4e62-b633-c760746c1a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:29.719704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebd5e0-1c58-4b40-99b8-0f73572f69db None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:29.721873 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1339/2676] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:29.724783 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3c57-6470-42df-8c5b-3285daad92d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:29.725641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3c57-6470-42df-8c5b-3285daad92d7 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.725929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3c57-6470-42df-8c5b-3285daad92d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.726209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3c57-6470-42df-8c5b-3285daad92d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.726370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5934-c64f-4e62-b633-c760746c1a37 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:29.736699 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb3c57-6470-42df-8c5b-3285daad92d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:29.737957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be5934-c64f-4e62-b633-c760746c1a37 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:29.738170 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb3c57-6470-42df-8c5b-3285daad92d7 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:29.745422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1f2a-f031-49b8-aa29-3bee914527a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:29.746128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1f2a-f031-49b8-aa29-3bee914527a0 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.746410 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1f2a-f031-49b8-aa29-3bee914527a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.755197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1f2a-f031-49b8-aa29-3bee914527a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.755197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1f2a-f031-49b8-aa29-3bee914527a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:29.755197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1f2a-f031-49b8-aa29-3bee914527a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:29.755509 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1338/2677] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:29.755603 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1f2a-f031-49b8-aa29-3bee914527a0 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:29.761898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1340/2678] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:06:29.766133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3c57-6470-42df-8c5b-3285daad92d7 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:29.776867 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3c57-6470-42df-8c5b-3285daad92d7 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:29.782429 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad244d-6735-4f76-9057-39b746131868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:29.788405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad244d-6735-4f76-9057-39b746131868 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.788405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad244d-6735-4f76-9057-39b746131868 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.788405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad244d-6735-4f76-9057-39b746131868 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.788405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad244d-6735-4f76-9057-39b746131868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:29.788405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad244d-6735-4f76-9057-39b746131868 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:29.875058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1339/2679] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:29 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:29.879983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1f2a-f031-49b8-aa29-3bee914527a0 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:29.897059 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1f2a-f031-49b8-aa29-3bee914527a0 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:29.965336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546020d-f640-45de-838a-1ed2067fc999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:29.971558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546020d-f640-45de-838a-1ed2067fc999 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.971558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546020d-f640-45de-838a-1ed2067fc999 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.971558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546020d-f640-45de-838a-1ed2067fc999 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.971558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546020d-f640-45de-838a-1ed2067fc999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:29.971558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546020d-f640-45de-838a-1ed2067fc999 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:30.101537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1341/2680] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:29 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:30.103452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad244d-6735-4f76-9057-39b746131868 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:30.122422 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad244d-6735-4f76-9057-39b746131868 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:30.132270 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1bc7f-8738-4f17-9f89-060819f86550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:30.133115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1bc7f-8738-4f17-9f89-060819f86550 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:30.133316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1bc7f-8738-4f17-9f89-060819f86550 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:30.133598 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1bc7f-8738-4f17-9f89-060819f86550 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:30.135056 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1bc7f-8738-4f17-9f89-060819f86550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:30.136818 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1bc7f-8738-4f17-9f89-060819f86550 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:30.148672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1340/2681] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:30.152829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546020d-f640-45de-838a-1ed2067fc999 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:30.176507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546020d-f640-45de-838a-1ed2067fc999 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:30.184846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb7a62-1973-491e-b202-981e96e8805c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:30.186062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb7a62-1973-491e-b202-981e96e8805c admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:30.186615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb7a62-1973-491e-b202-981e96e8805c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:30.187069 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb7a62-1973-491e-b202-981e96e8805c admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:30.188005 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb7a62-1973-491e-b202-981e96e8805c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:30.189877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb7a62-1973-491e-b202-981e96e8805c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:30.202650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1342/2682] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:30.206825 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1bc7f-8738-4f17-9f89-060819f86550 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:30.217475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1bc7f-8738-4f17-9f89-060819f86550 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:30.222147 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f44e01-4365-4f2c-baad-ae33a8ac73f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:30.223075 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f44e01-4365-4f2c-baad-ae33a8ac73f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:30.226580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f44e01-4365-4f2c-baad-ae33a8ac73f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:30.226580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f44e01-4365-4f2c-baad-ae33a8ac73f9 admin admin] PATH_INFO: `/v3/projects/d2c845effc6b41d6ae3e2b9d5fd8655b/users/0982d5ee2daf4c888fae4932d68b2da8/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:30.241003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f44e01-4365-4f2c-baad-ae33a8ac73f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c845effc6b41d6ae3e2b9d5fd8655b, user_id=0982d5ee2daf4c888fae4932d68b2da8, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:30.244031 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f44e01-4365-4f2c-baad-ae33a8ac73f9 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:30.354371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1341/2683] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:30 2022] PUT /identity/v3/projects/d2c845effc6b41d6ae3e2b9d5fd8655b/users/0982d5ee2daf4c888fae4932d68b2da8/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:30.365342 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df049ab-7e94-4793-8284-482519ffaf18 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:30.365643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df049ab-7e94-4793-8284-482519ffaf18 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:30.365930 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df049ab-7e94-4793-8284-482519ffaf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:30.393800 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-6df049ab-7e94-4793-8284-482519ffaf18 None None] MFA Rules not processed for user `0982d5ee2daf4c888fae4932d68b2da8`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:30.395112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6df049ab-7e94-4793-8284-482519ffaf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:30.406364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f44e01-4365-4f2c-baad-ae33a8ac73f9 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:30.410115 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e1f44e01-4365-4f2c-baad-ae33a8ac73f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:30.477845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1343/2684] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:30.482284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df049ab-7e94-4793-8284-482519ffaf18 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:30.504529 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f44e01-4365-4f2c-baad-ae33a8ac73f9 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:30.510086 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:30.511327 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:30.511803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:30.512135 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] PATH_INFO: `/v3/projects/8d9e75859a054f59ae8fd1902ab04459` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:30.513505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9e75859a054f59ae8fd1902ab04459)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:30.523813 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:30.531786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:30.538703 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:30.564810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df049ab-7e94-4793-8284-482519ffaf18 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:30.570720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d2df8-e5e9-493b-995a-f9cb484822ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:30.575682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d2df8-e5e9-493b-995a-f9cb484822ad admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:30.575682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d2df8-e5e9-493b-995a-f9cb484822ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:30.575682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d2df8-e5e9-493b-995a-f9cb484822ad admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:30.575682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d2df8-e5e9-493b-995a-f9cb484822ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:30.576676 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d2df8-e5e9-493b-995a-f9cb484822ad admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:30.669060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9e75859a054f59ae8fd1902ab04459', 'request_id': 'req-aa139108-4901-4599-8edc-ba9fd7430eca'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:30.676686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9e75859a054f59ae8fd1902ab04459', 'request_id': 'req-aa139108-4901-4599-8edc-ba9fd7430eca'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:30.683469 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9e75859a054f59ae8fd1902ab04459', 'request_id': 'req-aa139108-4901-4599-8edc-ba9fd7430eca'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:30.684889 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1344/2685] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:30 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:30.685823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d2df8-e5e9-493b-995a-f9cb484822ad admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:30.764695 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d2df8-e5e9-493b-995a-f9cb484822ad admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:30.770425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d70ee-b785-453f-9c25-64d270034a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:30.771167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d70ee-b785-453f-9c25-64d270034a14 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:30.776150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d70ee-b785-453f-9c25-64d270034a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:30.776150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d70ee-b785-453f-9c25-64d270034a14 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:30.776150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d70ee-b785-453f-9c25-64d270034a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:30.776150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d70ee-b785-453f-9c25-64d270034a14 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:30.803787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] The token cache is being invalidate because project 8d9e75859a054f59ae8fd1902ab04459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:30.809942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:30.809942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:30.809942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1342/2686] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:30 2022] DELETE /identity/v3/projects/8d9e75859a054f59ae8fd1902ab04459 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:30.811465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:30.813261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:30.891444 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa139108-4901-4599-8edc-ba9fd7430eca admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:30.896225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:30.897105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:30.897392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:30.897667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] PATH_INFO: `/v3/users/974c60c9a91d4624adad7bd7efb9795f` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:30.898932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=974c60c9a91d4624adad7bd7efb9795f)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:30.901461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:30.919696 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1345/2687] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:30 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:30.923531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d70ee-b785-453f-9c25-64d270034a14 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:30.926073 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-de2d70ee-b785-453f-9c25-64d270034a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:30.943614 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d70ee-b785-453f-9c25-64d270034a14 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:30.951182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b5409-3bc4-4196-8e0a-dde1c7b556dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:30.951995 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b5409-3bc4-4196-8e0a-dde1c7b556dc admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:30.952296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b5409-3bc4-4196-8e0a-dde1c7b556dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:30.952674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b5409-3bc4-4196-8e0a-dde1c7b556dc admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:30.953401 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b5409-3bc4-4196-8e0a-dde1c7b556dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:30.955032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b5409-3bc4-4196-8e0a-dde1c7b556dc admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:30.966354 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1346/2688] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:30.971016 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b5409-3bc4-4196-8e0a-dde1c7b556dc admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:30.982286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b5409-3bc4-4196-8e0a-dde1c7b556dc admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:30.990827 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c5440-2e35-4a45-bfb5-6b45313b0c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:30.992057 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c5440-2e35-4a45-bfb5-6b45313b0c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:30.993106 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c5440-2e35-4a45-bfb5-6b45313b0c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:30.993450 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c5440-2e35-4a45-bfb5-6b45313b0c8b admin admin] PATH_INFO: `/v3/projects/1da44df8169845b2a7391792f284ca17/users/7028440052284f34a77525bb3d3e7b06/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:31.011807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c5440-2e35-4a45-bfb5-6b45313b0c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da44df8169845b2a7391792f284ca17, user_id=7028440052284f34a77525bb3d3e7b06, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:31.015620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c5440-2e35-4a45-bfb5-6b45313b0c8b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:31.101836 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1347/2689] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:30 2022] PUT /identity/v3/projects/1da44df8169845b2a7391792f284ca17/users/7028440052284f34a77525bb3d3e7b06/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:31.105890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c5440-2e35-4a45-bfb5-6b45313b0c8b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:31.125308 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '974c60c9a91d4624adad7bd7efb9795f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:31.129451 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '974c60c9a91d4624adad7bd7efb9795f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:31.204599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '974c60c9a91d4624adad7bd7efb9795f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:31.207388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '974c60c9a91d4624adad7bd7efb9795f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:31.208340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '974c60c9a91d4624adad7bd7efb9795f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:31.229168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c5440-2e35-4a45-bfb5-6b45313b0c8b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:31.252667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227dd0d-9d4c-4cc0-9a24-87a2f3bf8964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:31.252667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227dd0d-9d4c-4cc0-9a24-87a2f3bf8964 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:31.252667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227dd0d-9d4c-4cc0-9a24-87a2f3bf8964 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:31.252667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227dd0d-9d4c-4cc0-9a24-87a2f3bf8964 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:31.253006 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227dd0d-9d4c-4cc0-9a24-87a2f3bf8964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:31.253085 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1343/2690] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:30 2022] DELETE /identity/v3/users/974c60c9a91d4624adad7bd7efb9795f => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:31.255883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227dd0d-9d4c-4cc0-9a24-87a2f3bf8964 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:31.270643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1348/2691] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:31.275675 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:31.277333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:31.377700 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1d932-dc1e-48b7-812d-ce7b188b8177 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:31.384681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77594bb4-cd48-4bb5-bc29-f176b7aa0500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:31.385647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77594bb4-cd48-4bb5-bc29-f176b7aa0500 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:31.385936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77594bb4-cd48-4bb5-bc29-f176b7aa0500 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:31.386282 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77594bb4-cd48-4bb5-bc29-f176b7aa0500 admin admin] PATH_INFO: `/v3/projects/1da44df8169845b2a7391792f284ca17/users/7028440052284f34a77525bb3d3e7b06/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:31.388625 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77594bb4-cd48-4bb5-bc29-f176b7aa0500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da44df8169845b2a7391792f284ca17, user_id=7028440052284f34a77525bb3d3e7b06, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:31.391661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77594bb4-cd48-4bb5-bc29-f176b7aa0500 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:31.407239 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1344/2692] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:31 2022] PUT /identity/v3/projects/1da44df8169845b2a7391792f284ca17/users/7028440052284f34a77525bb3d3e7b06/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:31.417618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9bb9c-ed09-43f8-b13a-4bb7840113cb None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:31.420061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9bb9c-ed09-43f8-b13a-4bb7840113cb None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:31.420061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9bb9c-ed09-43f8-b13a-4bb7840113cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:31.453127 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-97e9bb9c-ed09-43f8-b13a-4bb7840113cb None None] MFA Rules not processed for user `7028440052284f34a77525bb3d3e7b06`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:31.454347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-97e9bb9c-ed09-43f8-b13a-4bb7840113cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:31.500531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:31.552143 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1349/2693] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:31.570319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77594bb4-cd48-4bb5-bc29-f176b7aa0500 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:31.572638 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-77594bb4-cd48-4bb5-bc29-f176b7aa0500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:31.661648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77594bb4-cd48-4bb5-bc29-f176b7aa0500 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:31.666081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23430933-c74d-47a9-b68c-1aca420f31f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:31.668658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430933-c74d-47a9-b68c-1aca420f31f3 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:31.668658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430933-c74d-47a9-b68c-1aca420f31f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:31.668658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430933-c74d-47a9-b68c-1aca420f31f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:31.680844 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23430933-c74d-47a9-b68c-1aca420f31f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:31.682418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23430933-c74d-47a9-b68c-1aca420f31f3 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:31.725034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1345/2694] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:31.829635 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:31.973667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9bb9c-ed09-43f8-b13a-4bb7840113cb None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:31.975402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-97e9bb9c-ed09-43f8-b13a-4bb7840113cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:32.066371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9bb9c-ed09-43f8-b13a-4bb7840113cb None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:32.071936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:32.072810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:32.073091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:32.073367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] PATH_INFO: `/v3/projects/1c9671c64c4d421b89de458ef58b82f3` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:32.074580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c9671c64c4d421b89de458ef58b82f3)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:32.076546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:32.205432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c9671c64c4d421b89de458ef58b82f3', 'request_id': 'req-fed99ce4-3645-4e58-b534-a5408663f2fa'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:32.215605 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c9671c64c4d421b89de458ef58b82f3', 'request_id': 'req-fed99ce4-3645-4e58-b534-a5408663f2fa'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:32.221791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c9671c64c4d421b89de458ef58b82f3', 'request_id': 'req-fed99ce4-3645-4e58-b534-a5408663f2fa'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:32.301651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] The token cache is being invalidate because project 1c9671c64c4d421b89de458ef58b82f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:32.301651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:32.301651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-fed99ce4-3645-4e58-b534-a5408663f2fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:32.301651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1350/2695] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:31 2022] DELETE /identity/v3/projects/1c9671c64c4d421b89de458ef58b82f3 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:32.304250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039f35b-9e3c-4b2a-bb23-ab66a25ad57b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:32.304711 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039f35b-9e3c-4b2a-bb23-ab66a25ad57b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:32.304909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039f35b-9e3c-4b2a-bb23-ab66a25ad57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:32.327063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f039f35b-9e3c-4b2a-bb23-ab66a25ad57b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:32.328475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f039f35b-9e3c-4b2a-bb23-ab66a25ad57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:32.387362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1346/2696] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:32.395481 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106b8d5-3bd1-4522-892e-a1a2fdcae150 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:32.395769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106b8d5-3bd1-4522-892e-a1a2fdcae150 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:32.396046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106b8d5-3bd1-4522-892e-a1a2fdcae150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:32.419992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-7106b8d5-3bd1-4522-892e-a1a2fdcae150 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:32.421516 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-7106b8d5-3bd1-4522-892e-a1a2fdcae150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:32.437889 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1351/2697] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:32.442349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039f35b-9e3c-4b2a-bb23-ab66a25ad57b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:32.474794 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039f35b-9e3c-4b2a-bb23-ab66a25ad57b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:32.481793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18173513-3778-42e6-a7a3-eaae2e91fd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:32.483417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18173513-3778-42e6-a7a3-eaae2e91fd23 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:32.483806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18173513-3778-42e6-a7a3-eaae2e91fd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:32.487029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18173513-3778-42e6-a7a3-eaae2e91fd23 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:32.487029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18173513-3778-42e6-a7a3-eaae2e91fd23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:32.487029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18173513-3778-42e6-a7a3-eaae2e91fd23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:32.488951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18173513-3778-42e6-a7a3-eaae2e91fd23 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:32.512075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106b8d5-3bd1-4522-892e-a1a2fdcae150 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:32.520806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1347/2698] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:32.525366 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106b8d5-3bd1-4522-892e-a1a2fdcae150 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:32.532703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06092087-e38e-4b0d-a6cf-2f0164cfb242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:32.533678 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06092087-e38e-4b0d-a6cf-2f0164cfb242 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:32.534095 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06092087-e38e-4b0d-a6cf-2f0164cfb242 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:32.534586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06092087-e38e-4b0d-a6cf-2f0164cfb242 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:32.535389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06092087-e38e-4b0d-a6cf-2f0164cfb242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:32.538056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06092087-e38e-4b0d-a6cf-2f0164cfb242 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:32.634616 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:32.645741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1352/2699] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:32 2022] POST /identity/v3/projects => generated 389 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:32.650672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18173513-3778-42e6-a7a3-eaae2e91fd23 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:32.663361 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18173513-3778-42e6-a7a3-eaae2e91fd23 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:32.692259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:32.736806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111bb3d-4b42-435a-a68c-1606b71dd6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:32.738021 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111bb3d-4b42-435a-a68c-1606b71dd6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:32.739735 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111bb3d-4b42-435a-a68c-1606b71dd6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:32.740301 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111bb3d-4b42-435a-a68c-1606b71dd6c7 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:32.742288 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111bb3d-4b42-435a-a68c-1606b71dd6c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:32.746863 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111bb3d-4b42-435a-a68c-1606b71dd6c7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:32.859500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1348/2700] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:32 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:32.861044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06092087-e38e-4b0d-a6cf-2f0164cfb242 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:32.879242 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111bb3d-4b42-435a-a68c-1606b71dd6c7 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:32.881120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1111bb3d-4b42-435a-a68c-1606b71dd6c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:32.900872 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111bb3d-4b42-435a-a68c-1606b71dd6c7 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:32.901218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06092087-e38e-4b0d-a6cf-2f0164cfb242 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:32.908496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:32.911220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:32.911489 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:32.911914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] PATH_INFO: `/v3/users/0982d5ee2daf4c888fae4932d68b2da8` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:32.913574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac60dc2-99d2-4a74-9842-42905c67961f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:32.914269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac60dc2-99d2-4a74-9842-42905c67961f admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:32.914606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac60dc2-99d2-4a74-9842-42905c67961f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:32.915763 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac60dc2-99d2-4a74-9842-42905c67961f admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:32.916309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0982d5ee2daf4c888fae4932d68b2da8)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:32.916896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac60dc2-99d2-4a74-9842-42905c67961f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:32.919128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac60dc2-99d2-4a74-9842-42905c67961f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:32.919596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:32.935026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1353/2701] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:32.939968 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac60dc2-99d2-4a74-9842-42905c67961f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:32.953376 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac60dc2-99d2-4a74-9842-42905c67961f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:32.958377 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f05b1-3d1c-4b54-b7d7-a0bd0fdaf173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:32.959215 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f05b1-3d1c-4b54-b7d7-a0bd0fdaf173 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:32.959585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f05b1-3d1c-4b54-b7d7-a0bd0fdaf173 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:32.959953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f05b1-3d1c-4b54-b7d7-a0bd0fdaf173 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:32.960823 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f05b1-3d1c-4b54-b7d7-a0bd0fdaf173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:32.962729 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f05b1-3d1c-4b54-b7d7-a0bd0fdaf173 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:32.973929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1354/2702] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:32.978855 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f05b1-3d1c-4b54-b7d7-a0bd0fdaf173 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:32.988898 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f05b1-3d1c-4b54-b7d7-a0bd0fdaf173 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:32.994781 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f85b1a-aaa4-4e71-85fd-70eb4a1c84a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:32.999665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f85b1a-aaa4-4e71-85fd-70eb4a1c84a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:32.999665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f85b1a-aaa4-4e71-85fd-70eb4a1c84a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:32.999665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f85b1a-aaa4-4e71-85fd-70eb4a1c84a3 admin admin] PATH_INFO: `/v3/projects/6912226954a64930902cd6247f804746/users/2559073865874a69b579c2c4b8c67781/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:33.011374 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f85b1a-aaa4-4e71-85fd-70eb4a1c84a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6912226954a64930902cd6247f804746, user_id=2559073865874a69b579c2c4b8c67781, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:33.014076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f85b1a-aaa4-4e71-85fd-70eb4a1c84a3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:33.111699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1355/2703] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:32 2022] PUT /identity/v3/projects/6912226954a64930902cd6247f804746/users/2559073865874a69b579c2c4b8c67781/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:33.119957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857d017-0839-4dcf-ac66-9dc7f60d8899 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:33.119957 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857d017-0839-4dcf-ac66-9dc7f60d8899 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:33.120442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857d017-0839-4dcf-ac66-9dc7f60d8899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:33.152289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0982d5ee2daf4c888fae4932d68b2da8'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:33.160719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0982d5ee2daf4c888fae4932d68b2da8'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:33.162933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e857d017-0839-4dcf-ac66-9dc7f60d8899 None None] MFA Rules not processed for user `2559073865874a69b579c2c4b8c67781`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:33.164320 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e857d017-0839-4dcf-ac66-9dc7f60d8899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:33.247488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1356/2704] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:33.254923 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857d017-0839-4dcf-ac66-9dc7f60d8899 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:33.255253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0982d5ee2daf4c888fae4932d68b2da8'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:33.256169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0982d5ee2daf4c888fae4932d68b2da8'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:33.258156 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-37f9585f-1dbe-4400-b84d-f84b45fcd609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0982d5ee2daf4c888fae4932d68b2da8'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:33.289511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1349/2705] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:32 2022] DELETE /identity/v3/users/0982d5ee2daf4c888fae4932d68b2da8 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:33.297310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6672f-1935-47ce-a5b3-a91bc2b9074a None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:33.299972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6672f-1935-47ce-a5b3-a91bc2b9074a None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:33.299972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6672f-1935-47ce-a5b3-a91bc2b9074a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:33.335877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-27e6672f-1935-47ce-a5b3-a91bc2b9074a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:33.338408 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-27e6672f-1935-47ce-a5b3-a91bc2b9074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:33.375660 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857d017-0839-4dcf-ac66-9dc7f60d8899 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:33.442071 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1350/2706] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:33.455897 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6672f-1935-47ce-a5b3-a91bc2b9074a None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:33.458609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-27e6672f-1935-47ce-a5b3-a91bc2b9074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:33.464745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf0f90-e4ba-4676-9678-4981dba2032c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:33.465746 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0f90-e4ba-4676-9678-4981dba2032c admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:33.466233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0f90-e4ba-4676-9678-4981dba2032c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:33.466581 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0f90-e4ba-4676-9678-4981dba2032c admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:33.467567 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf0f90-e4ba-4676-9678-4981dba2032c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:33.469607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf0f90-e4ba-4676-9678-4981dba2032c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:33.509432 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:33.540578 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6672f-1935-47ce-a5b3-a91bc2b9074a None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:33.546920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1933a-e061-4128-9bee-3ae6d479d6ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:33.547590 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1933a-e061-4128-9bee-3ae6d479d6ff service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:33.548017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1933a-e061-4128-9bee-3ae6d479d6ff service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:33.548302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1933a-e061-4128-9bee-3ae6d479d6ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:33.564113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1933a-e061-4128-9bee-3ae6d479d6ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:33.566888 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1933a-e061-4128-9bee-3ae6d479d6ff service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:33.583802 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1357/2707] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:33 2022] POST /identity/v3/projects => generated 387 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:33.587790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aca0b-0995-451e-ad89-386721f27778 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:33.588110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aca0b-0995-451e-ad89-386721f27778 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:33.588362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4aca0b-0995-451e-ad89-386721f27778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:33.600729 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1351/2708] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:33.606248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1933a-e061-4128-9bee-3ae6d479d6ff service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:33.612315 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8cf1933a-e061-4128-9bee-3ae6d479d6ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:33.644233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1933a-e061-4128-9bee-3ae6d479d6ff service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:33.671813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-6b4aca0b-0995-451e-ad89-386721f27778 None None] MFA Rules not processed for user `a01a4b9868714295a1000e0a5a81143b`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:33.673965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6b4aca0b-0995-451e-ad89-386721f27778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:33.745744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c5d39-0ffc-49d3-8e4b-4c050d52a830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:33.746453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c5d39-0ffc-49d3-8e4b-4c050d52a830 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:33.746819 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c5d39-0ffc-49d3-8e4b-4c050d52a830 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:33.747553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c5d39-0ffc-49d3-8e4b-4c050d52a830 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:33.747911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c5d39-0ffc-49d3-8e4b-4c050d52a830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:33.749895 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c5d39-0ffc-49d3-8e4b-4c050d52a830 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:33.830570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1358/2709] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:33.836524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4aca0b-0995-451e-ad89-386721f27778 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:33.839206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6b4aca0b-0995-451e-ad89-386721f27778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:33.854039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4aca0b-0995-451e-ad89-386721f27778 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:33.858911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4ea1b-c06e-4456-b6e1-9e9b0e04a00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:33.859770 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4ea1b-c06e-4456-b6e1-9e9b0e04a00e admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:33.860053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4ea1b-c06e-4456-b6e1-9e9b0e04a00e admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:33.860333 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4ea1b-c06e-4456-b6e1-9e9b0e04a00e admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:33.861039 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4ea1b-c06e-4456-b6e1-9e9b0e04a00e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:33.863014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4ea1b-c06e-4456-b6e1-9e9b0e04a00e admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:33.891074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:33.901192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1352/2710] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:33 2022] POST /identity/v3/users => generated 300 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:33.907811 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c5d39-0ffc-49d3-8e4b-4c050d52a830 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:33.931089 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c5d39-0ffc-49d3-8e4b-4c050d52a830 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:33.937929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffaf8b-287c-4d48-9a65-a2aab2f9ba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:33.938885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffaf8b-287c-4d48-9a65-a2aab2f9ba23 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:33.939171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffaf8b-287c-4d48-9a65-a2aab2f9ba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:33.939510 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffaf8b-287c-4d48-9a65-a2aab2f9ba23 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:33.940236 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffaf8b-287c-4d48-9a65-a2aab2f9ba23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:33.942188 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffaf8b-287c-4d48-9a65-a2aab2f9ba23 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:33.956596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1353/2711] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:33.958024 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffaf8b-287c-4d48-9a65-a2aab2f9ba23 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:33.970018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffaf8b-287c-4d48-9a65-a2aab2f9ba23 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:33.975493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbaf9c-6fc0-4f5f-819f-8b0a440b626a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:33.976679 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbaf9c-6fc0-4f5f-819f-8b0a440b626a admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:33.976960 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbaf9c-6fc0-4f5f-819f-8b0a440b626a admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:33.977261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbaf9c-6fc0-4f5f-819f-8b0a440b626a admin admin] PATH_INFO: `/v3/projects/0de3dbd984334a9e9842c8756ef37a9b/users/468f23aae137453db1843835b277521a/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:33.990834 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1359/2712] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:33 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:33.995990 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4ea1b-c06e-4456-b6e1-9e9b0e04a00e admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:34.001976 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbaf9c-6fc0-4f5f-819f-8b0a440b626a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de3dbd984334a9e9842c8756ef37a9b, user_id=468f23aae137453db1843835b277521a, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:34.006689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbaf9c-6fc0-4f5f-819f-8b0a440b626a admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:34.084965 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4ea1b-c06e-4456-b6e1-9e9b0e04a00e admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:34.090259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920b11e-a5e8-4273-83ed-52f98e304a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:34.091064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b11e-a5e8-4273-83ed-52f98e304a93 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:34.091422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b11e-a5e8-4273-83ed-52f98e304a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:34.095783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b11e-a5e8-4273-83ed-52f98e304a93 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:34.095783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920b11e-a5e8-4273-83ed-52f98e304a93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:34.095783 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920b11e-a5e8-4273-83ed-52f98e304a93 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:34.122220 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1354/2713] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:33 2022] PUT /identity/v3/projects/0de3dbd984334a9e9842c8756ef37a9b/users/468f23aae137453db1843835b277521a/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:34.124277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbaf9c-6fc0-4f5f-819f-8b0a440b626a admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:34.192796 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbaf9c-6fc0-4f5f-819f-8b0a440b626a admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:34.199241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a248b6e-f706-4db8-98b0-5c29b8e2aec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:34.200189 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a248b6e-f706-4db8-98b0-5c29b8e2aec9 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:34.200503 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a248b6e-f706-4db8-98b0-5c29b8e2aec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:34.200952 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a248b6e-f706-4db8-98b0-5c29b8e2aec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:34.201754 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a248b6e-f706-4db8-98b0-5c29b8e2aec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:34.204516 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a248b6e-f706-4db8-98b0-5c29b8e2aec9 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:34.215843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1355/2714] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:34.216680 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a248b6e-f706-4db8-98b0-5c29b8e2aec9 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:34.218394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4a248b6e-f706-4db8-98b0-5c29b8e2aec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:34.225507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1360/2715] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:33 2022] POST /identity/v3/users => generated 298 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:34.226502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920b11e-a5e8-4273-83ed-52f98e304a93 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:34.237460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920b11e-a5e8-4273-83ed-52f98e304a93 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:34.243573 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c47ecb-9250-4a88-90c5-b7c6fad6d199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:34.244469 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c47ecb-9250-4a88-90c5-b7c6fad6d199 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:34.244748 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c47ecb-9250-4a88-90c5-b7c6fad6d199 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:34.245018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c47ecb-9250-4a88-90c5-b7c6fad6d199 admin admin] PATH_INFO: `/v3/projects/0de3dbd984334a9e9842c8756ef37a9b/users/468f23aae137453db1843835b277521a/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:34.247186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c47ecb-9250-4a88-90c5-b7c6fad6d199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de3dbd984334a9e9842c8756ef37a9b, user_id=468f23aae137453db1843835b277521a, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:34.249797 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c47ecb-9250-4a88-90c5-b7c6fad6d199 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:34.265193 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a248b6e-f706-4db8-98b0-5c29b8e2aec9 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:34.267852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1361/2716] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:34 2022] PUT /identity/v3/projects/0de3dbd984334a9e9842c8756ef37a9b/users/468f23aae137453db1843835b277521a/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:34.268862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c47ecb-9250-4a88-90c5-b7c6fad6d199 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:34.342795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:34.343661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:34.343938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:34.344207 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] PATH_INFO: `/v3/projects/d2c845effc6b41d6ae3e2b9d5fd8655b` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:34.345542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2c845effc6b41d6ae3e2b9d5fd8655b)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:34.347476 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:34.365786 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c47ecb-9250-4a88-90c5-b7c6fad6d199 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:34.371185 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded9fb9-c6aa-4c0b-8379-2b898fbf66bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:34.371860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9fb9-c6aa-4c0b-8379-2b898fbf66bc admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:34.372144 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9fb9-c6aa-4c0b-8379-2b898fbf66bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:34.372470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded9fb9-c6aa-4c0b-8379-2b898fbf66bc admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:34.373148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded9fb9-c6aa-4c0b-8379-2b898fbf66bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:34.374818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded9fb9-c6aa-4c0b-8379-2b898fbf66bc admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:34.385189 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1362/2717] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:34.388355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a9b7c-2237-4c99-bec8-57ec9ca72190 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:34.388355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a9b7c-2237-4c99-bec8-57ec9ca72190 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:34.388355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a9b7c-2237-4c99-bec8-57ec9ca72190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:34.462768 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2c845effc6b41d6ae3e2b9d5fd8655b', 'request_id': 'req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:34.470134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2c845effc6b41d6ae3e2b9d5fd8655b', 'request_id': 'req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:34.478493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2c845effc6b41d6ae3e2b9d5fd8655b', 'request_id': 'req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:34.532852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:34.548676 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8d3a9b7c-2237-4c99-bec8-57ec9ca72190 None None] MFA Rules not processed for user `468f23aae137453db1843835b277521a`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:34.549998 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8d3a9b7c-2237-4c99-bec8-57ec9ca72190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:34.569184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] The token cache is being invalidate because project d2c845effc6b41d6ae3e2b9d5fd8655b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:34.569719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:34.571363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:34.572117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:34.573837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1356/2718] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:34 2022] DELETE /identity/v3/projects/d2c845effc6b41d6ae3e2b9d5fd8655b => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:34.577732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:34.579450 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:34.625779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1363/2719] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:34.626894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a9b7c-2237-4c99-bec8-57ec9ca72190 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:34.628974 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8d3a9b7c-2237-4c99-bec8-57ec9ca72190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:34.654705 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeb36c-dd91-41fc-b06c-44b1d3c26986 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:34.659663 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaffbe0-37e3-4b89-b423-2f63420e1665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:34.664063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaffbe0-37e3-4b89-b423-2f63420e1665 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:34.664063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaffbe0-37e3-4b89-b423-2f63420e1665 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:34.664063 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaffbe0-37e3-4b89-b423-2f63420e1665 admin admin] PATH_INFO: `/v3/projects/61334b1f57eb4f599673de8b21300e1e/users/a7d69a23418e491f883936f409067f26/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:34.676739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaffbe0-37e3-4b89-b423-2f63420e1665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61334b1f57eb4f599673de8b21300e1e, user_id=a7d69a23418e491f883936f409067f26, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:34.679581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaffbe0-37e3-4b89-b423-2f63420e1665 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:34.704266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a9b7c-2237-4c99-bec8-57ec9ca72190 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:34.708858 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:34.709732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:34.710010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:34.710296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] PATH_INFO: `/v3/users/7028440052284f34a77525bb3d3e7b06` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:34.712121 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7028440052284f34a77525bb3d3e7b06)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:34.714065 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:34.770553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1357/2720] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:34 2022] PUT /identity/v3/projects/61334b1f57eb4f599673de8b21300e1e/users/a7d69a23418e491f883936f409067f26/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:34.776839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaffbe0-37e3-4b89-b423-2f63420e1665 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:34.776839 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0aaffbe0-37e3-4b89-b423-2f63420e1665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:34.850397 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaffbe0-37e3-4b89-b423-2f63420e1665 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:34.855443 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cc2c2-1ed9-41a8-bb86-40d8340dff52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:34.856257 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cc2c2-1ed9-41a8-bb86-40d8340dff52 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:34.856886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cc2c2-1ed9-41a8-bb86-40d8340dff52 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:34.856886 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cc2c2-1ed9-41a8-bb86-40d8340dff52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:34.943486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cc2c2-1ed9-41a8-bb86-40d8340dff52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:34.945027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cc2c2-1ed9-41a8-bb86-40d8340dff52 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:34.954324 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7028440052284f34a77525bb3d3e7b06'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:34.962707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1358/2721] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:34.964870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cc2c2-1ed9-41a8-bb86-40d8340dff52 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:34.968570 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7028440052284f34a77525bb3d3e7b06'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:34.970155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7028440052284f34a77525bb3d3e7b06'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:35.045389 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7028440052284f34a77525bb3d3e7b06'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:35.046381 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7028440052284f34a77525bb3d3e7b06'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:35.066091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cc2c2-1ed9-41a8-bb86-40d8340dff52 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:35.068670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:35.072835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a422e1a-74e6-4009-8638-91803c9dc665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:35.073444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1364/2722] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:34 2022] DELETE /identity/v3/users/7028440052284f34a77525bb3d3e7b06 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:35.073596 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a422e1a-74e6-4009-8638-91803c9dc665 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:35.073877 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a422e1a-74e6-4009-8638-91803c9dc665 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:35.074152 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a422e1a-74e6-4009-8638-91803c9dc665 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:35.074923 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a422e1a-74e6-4009-8638-91803c9dc665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:35.080559 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a422e1a-74e6-4009-8638-91803c9dc665 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:35.090309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1359/2723] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:35.094576 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:35.096112 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:35.155211 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a422e1a-74e6-4009-8638-91803c9dc665 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:35.157144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-8a422e1a-74e6-4009-8638-91803c9dc665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:35.168109 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:35.177241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a422e1a-74e6-4009-8638-91803c9dc665 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:35.199554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ea4fd-eb98-4897-997c-5bb11b5fdf32 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:35.204830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0f9cf-20e9-4af6-8009-9ab0fab11cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:35.205871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0f9cf-20e9-4af6-8009-9ab0fab11cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:35.206216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0f9cf-20e9-4af6-8009-9ab0fab11cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:35.206619 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0f9cf-20e9-4af6-8009-9ab0fab11cbc admin admin] PATH_INFO: `/v3/projects/61334b1f57eb4f599673de8b21300e1e/users/a7d69a23418e491f883936f409067f26/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:35.208806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0f9cf-20e9-4af6-8009-9ab0fab11cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=61334b1f57eb4f599673de8b21300e1e, user_id=a7d69a23418e491f883936f409067f26, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:35.212255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0f9cf-20e9-4af6-8009-9ab0fab11cbc admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:35.229031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1365/2724] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:35 2022] PUT /identity/v3/projects/61334b1f57eb4f599673de8b21300e1e/users/a7d69a23418e491f883936f409067f26/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:35.252237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:35.259197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:35.259197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:35.259197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] PATH_INFO: `/v3/users/2559073865874a69b579c2c4b8c67781` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:35.259197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=2559073865874a69b579c2c4b8c67781)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:35.259197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:35.460306 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2559073865874a69b579c2c4b8c67781'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:35.468989 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2559073865874a69b579c2c4b8c67781'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:35.529737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2559073865874a69b579c2c4b8c67781'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:35.531806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2559073865874a69b579c2c4b8c67781'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:35.532485 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2559073865874a69b579c2c4b8c67781'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:35.552632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1360/2725] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:35 2022] DELETE /identity/v3/users/2559073865874a69b579c2c4b8c67781 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:35.557956 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db311d-f3d4-452a-a449-a86add636eb9 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:35.558247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db311d-f3d4-452a-a449-a86add636eb9 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:35.558683 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db311d-f3d4-452a-a449-a86add636eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:35.604939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a3db311d-f3d4-452a-a449-a86add636eb9 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:35.606157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a3db311d-f3d4-452a-a449-a86add636eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:35.634633 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:35.636124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:35.709655 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:35.755180 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1366/2726] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:35.759692 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70150755-d0e2-495c-8d8e-223d7db88fde admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:35.762665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db311d-f3d4-452a-a449-a86add636eb9 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:35.767657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a3db311d-f3d4-452a-a449-a86add636eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:35.773418 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:35.774385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:35.774698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:35.774975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] PATH_INFO: `/v3/projects/1da44df8169845b2a7391792f284ca17` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:35.779334 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da44df8169845b2a7391792f284ca17)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:35.781275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:35.781521 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:35.871040 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db311d-f3d4-452a-a449-a86add636eb9 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:35.874053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c419621-7926-44d7-b008-5917858f8c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:35.874954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419621-7926-44d7-b008-5917858f8c07 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:35.875244 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419621-7926-44d7-b008-5917858f8c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:35.875815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c419621-7926-44d7-b008-5917858f8c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:35.887666 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c419621-7926-44d7-b008-5917858f8c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:35.889237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c419621-7926-44d7-b008-5917858f8c07 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:35.910884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1367/2727] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:35.945571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da44df8169845b2a7391792f284ca17', 'request_id': 'req-3edae555-8476-4823-a564-bfc2a5a17d72'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:35.957091 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da44df8169845b2a7391792f284ca17', 'request_id': 'req-3edae555-8476-4823-a564-bfc2a5a17d72'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:35.964023 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da44df8169845b2a7391792f284ca17', 'request_id': 'req-3edae555-8476-4823-a564-bfc2a5a17d72'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:36.072898 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] The token cache is being invalidate because project 1da44df8169845b2a7391792f284ca17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:36.073183 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:36.074003 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3edae555-8476-4823-a564-bfc2a5a17d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:36.075666 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1361/2728] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:35 2022] DELETE /identity/v3/projects/1da44df8169845b2a7391792f284ca17 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:36.081865 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06fa4a-600f-4cb7-ba78-1c73c9ce887a None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:36.082158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06fa4a-600f-4cb7-ba78-1c73c9ce887a None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:36.082779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06fa4a-600f-4cb7-ba78-1c73c9ce887a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:36.086162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:36.186431 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-2e06fa4a-600f-4cb7-ba78-1c73c9ce887a None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:36.187862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2e06fa4a-600f-4cb7-ba78-1c73c9ce887a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:36.280117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1368/2729] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:36.289268 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39a423-6758-440c-9ac7-f5e3de7d4ac4 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:36.290507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39a423-6758-440c-9ac7-f5e3de7d4ac4 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:36.291169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39a423-6758-440c-9ac7-f5e3de7d4ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:36.317399 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1c39a423-6758-440c-9ac7-f5e3de7d4ac4 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:36.319363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1c39a423-6758-440c-9ac7-f5e3de7d4ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:36.335791 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1362/2730] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:36.338370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06fa4a-600f-4cb7-ba78-1c73c9ce887a None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:36.350521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06fa4a-600f-4cb7-ba78-1c73c9ce887a None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:36.356629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6eb59-a4d1-41da-952c-b894f9d03c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:36.357049 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6eb59-a4d1-41da-952c-b894f9d03c7c admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:36.357336 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6eb59-a4d1-41da-952c-b894f9d03c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:36.357615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6eb59-a4d1-41da-952c-b894f9d03c7c admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:36.358355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6eb59-a4d1-41da-952c-b894f9d03c7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:36.358699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6eb59-a4d1-41da-952c-b894f9d03c7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:36.360742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6eb59-a4d1-41da-952c-b894f9d03c7c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:36.390587 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39a423-6758-440c-9ac7-f5e3de7d4ac4 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:36.398857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1369/2731] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:36.404868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39a423-6758-440c-9ac7-f5e3de7d4ac4 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:36.413228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ef70-6db8-4785-ab80-ad3f2f24f987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:36.413916 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ef70-6db8-4785-ab80-ad3f2f24f987 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:36.414201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ef70-6db8-4785-ab80-ad3f2f24f987 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:36.414500 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ef70-6db8-4785-ab80-ad3f2f24f987 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:36.415487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91ef70-6db8-4785-ab80-ad3f2f24f987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:36.421000 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91ef70-6db8-4785-ab80-ad3f2f24f987 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:36.550289 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1363/2732] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:36 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:36.554558 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6eb59-a4d1-41da-952c-b894f9d03c7c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:36.561434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ef70-6db8-4785-ab80-ad3f2f24f987 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:36.567717 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ce91ef70-6db8-4785-ab80-ad3f2f24f987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:36.568413 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6eb59-a4d1-41da-952c-b894f9d03c7c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:36.675313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ef70-6db8-4785-ab80-ad3f2f24f987 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:36.678760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae8695-2df7-41db-8883-b64a8d064e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:36.679650 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae8695-2df7-41db-8883-b64a8d064e27 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:36.679983 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae8695-2df7-41db-8883-b64a8d064e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:36.680828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae8695-2df7-41db-8883-b64a8d064e27 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:36.681217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:36.681613 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae8695-2df7-41db-8883-b64a8d064e27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:36.682225 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:36.682607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:36.682951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] PATH_INFO: `/v3/projects/6912226954a64930902cd6247f804746` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:36.683708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae8695-2df7-41db-8883-b64a8d064e27 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:36.684471 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6912226954a64930902cd6247f804746)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:36.686638 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:36.755745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1370/2733] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:36 2022] POST /identity/v3/users => generated 284 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:36.761273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae8695-2df7-41db-8883-b64a8d064e27 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:36.779674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae8695-2df7-41db-8883-b64a8d064e27 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:36.799282 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f91d13-591a-454a-9fae-9ba9441baaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:36.800043 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f91d13-591a-454a-9fae-9ba9441baaae admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:36.800329 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f91d13-591a-454a-9fae-9ba9441baaae admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:36.800845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f91d13-591a-454a-9fae-9ba9441baaae admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:36.804684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f91d13-591a-454a-9fae-9ba9441baaae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:36.804684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f91d13-591a-454a-9fae-9ba9441baaae admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:36.825119 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1371/2734] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:36.826448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f91d13-591a-454a-9fae-9ba9441baaae admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:36.828080 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6912226954a64930902cd6247f804746', 'request_id': 'req-9c9685bf-8501-4f67-a2f1-c320a09177b8'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:36.839333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6912226954a64930902cd6247f804746', 'request_id': 'req-9c9685bf-8501-4f67-a2f1-c320a09177b8'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:36.861233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f91d13-591a-454a-9fae-9ba9441baaae admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:36.862410 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6912226954a64930902cd6247f804746', 'request_id': 'req-9c9685bf-8501-4f67-a2f1-c320a09177b8'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:36.901617 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:36.947914 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6a4a-4f12-486c-881b-bf76c2f982ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:36.948470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6a4a-4f12-486c-881b-bf76c2f982ee admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:36.948760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6a4a-4f12-486c-881b-bf76c2f982ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:36.949041 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad6a4a-4f12-486c-881b-bf76c2f982ee admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:36.958278 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad6a4a-4f12-486c-881b-bf76c2f982ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:36.959309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] The token cache is being invalidate because project 6912226954a64930902cd6247f804746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:36.959429 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad6a4a-4f12-486c-881b-bf76c2f982ee admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:36.959534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:36.959534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:36.965227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1364/2735] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:36 2022] DELETE /identity/v3/projects/6912226954a64930902cd6247f804746 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:36.966667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:36.970764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1372/2736] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:36.970926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:36.971218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6a4a-4f12-486c-881b-bf76c2f982ee admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:36.973692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-eaad6a4a-4f12-486c-881b-bf76c2f982ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:36.985146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9685bf-8501-4f67-a2f1-c320a09177b8 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:36.991928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:36.994051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:36.994051 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:36.994550 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] PATH_INFO: `/v3/users/468f23aae137453db1843835b277521a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:36.996805 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] RBAC: Authorizing `identity:delete_user(user_id=468f23aae137453db1843835b277521a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:36.999244 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:37.011443 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad6a4a-4f12-486c-881b-bf76c2f982ee admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:37.025703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b44400-a57e-4c22-a824-1017a767c8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:37.026806 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b44400-a57e-4c22-a824-1017a767c8a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:37.027153 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b44400-a57e-4c22-a824-1017a767c8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:37.028390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b44400-a57e-4c22-a824-1017a767c8a9 admin admin] PATH_INFO: `/v3/projects/91a61dfae6db4d46878906d2038f2a6d/users/8d9c651be44a47fc81d80a2448682e12/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:37.054871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b44400-a57e-4c22-a824-1017a767c8a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a61dfae6db4d46878906d2038f2a6d, user_id=8d9c651be44a47fc81d80a2448682e12, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:37.059027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b44400-a57e-4c22-a824-1017a767c8a9 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:37.108022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1373/2737] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:36 2022] PUT /identity/v3/projects/91a61dfae6db4d46878906d2038f2a6d/users/8d9c651be44a47fc81d80a2448682e12/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:37.124219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026d69-fdf5-4113-814f-927d936dcf09 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:37.125092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026d69-fdf5-4113-814f-927d936dcf09 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:37.130758 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026d69-fdf5-4113-814f-927d936dcf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:37.144388 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468f23aae137453db1843835b277521a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:37.155302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468f23aae137453db1843835b277521a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:37.163470 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-e9026d69-fdf5-4113-814f-927d936dcf09 None None] MFA Rules not processed for user `8d9c651be44a47fc81d80a2448682e12`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:37.164879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e9026d69-fdf5-4113-814f-927d936dcf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:37.205059 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468f23aae137453db1843835b277521a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:37.206005 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468f23aae137453db1843835b277521a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:37.207439 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468f23aae137453db1843835b277521a'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:37.234203 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1365/2738] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:36 2022] DELETE /identity/v3/users/468f23aae137453db1843835b277521a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:37.265578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1374/2739] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:37.266137 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:37.267787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:37.357477 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c886d-439f-4dcc-8732-d10e820c119f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:37.363890 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb340c-6ce5-4345-9bac-0799f1f293b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:37.364573 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb340c-6ce5-4345-9bac-0799f1f293b0 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:37.365259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb340c-6ce5-4345-9bac-0799f1f293b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:37.365259 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb340c-6ce5-4345-9bac-0799f1f293b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:37.365879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb340c-6ce5-4345-9bac-0799f1f293b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:37.368906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb340c-6ce5-4345-9bac-0799f1f293b0 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:37.489837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:37.504581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1366/2740] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:37 2022] POST /identity/v3/projects => generated 367 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:37.506993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9026d69-fdf5-4113-814f-927d936dcf09 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:37.595179 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9026d69-fdf5-4113-814f-927d936dcf09 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:37.676088 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06918fc6-de23-4e11-b82a-149e91dceb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:37.676780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06918fc6-de23-4e11-b82a-149e91dceb6b admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:37.677062 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06918fc6-de23-4e11-b82a-149e91dceb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:37.677356 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06918fc6-de23-4e11-b82a-149e91dceb6b admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:37.678138 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06918fc6-de23-4e11-b82a-149e91dceb6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:37.680235 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06918fc6-de23-4e11-b82a-149e91dceb6b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:37.815238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1375/2741] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:37 2022] POST /identity/v3/users => generated 290 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:37.817980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb340c-6ce5-4345-9bac-0799f1f293b0 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:37.835701 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:37.841235 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb340c-6ce5-4345-9bac-0799f1f293b0 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:37.846125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f5d54-27cc-4396-882d-4a1bf6cd6e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:37.846963 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f5d54-27cc-4396-882d-4a1bf6cd6e61 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:37.847257 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f5d54-27cc-4396-882d-4a1bf6cd6e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:37.847527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f5d54-27cc-4396-882d-4a1bf6cd6e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:37.848362 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f5d54-27cc-4396-882d-4a1bf6cd6e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:37.851484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f5d54-27cc-4396-882d-4a1bf6cd6e61 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:37.869405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1367/2742] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:37.870125 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06918fc6-de23-4e11-b82a-149e91dceb6b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:37.887130 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06918fc6-de23-4e11-b82a-149e91dceb6b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:37.899856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e00530-9f15-4c8b-a1c6-7dd03990ad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:37.900780 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e00530-9f15-4c8b-a1c6-7dd03990ad4f admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:37.901061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e00530-9f15-4c8b-a1c6-7dd03990ad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:37.901946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e00530-9f15-4c8b-a1c6-7dd03990ad4f admin admin] PATH_INFO: `/v3/projects/c9a98126fca5457999d2fbaa0fb66cbb/users/d01db669717841059bd13216750e5597/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:37.932720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e00530-9f15-4c8b-a1c6-7dd03990ad4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a98126fca5457999d2fbaa0fb66cbb, user_id=d01db669717841059bd13216750e5597, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:37.939538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e00530-9f15-4c8b-a1c6-7dd03990ad4f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:38.020298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1376/2743] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:37 2022] PUT /identity/v3/projects/c9a98126fca5457999d2fbaa0fb66cbb/users/d01db669717841059bd13216750e5597/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:38.024515 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f5d54-27cc-4396-882d-4a1bf6cd6e61 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:38.037582 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f5d54-27cc-4396-882d-4a1bf6cd6e61 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:38.103238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e00530-9f15-4c8b-a1c6-7dd03990ad4f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:38.105346 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-35e00530-9f15-4c8b-a1c6-7dd03990ad4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:38.123564 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffefef5-0355-42bd-9d01-2c3bbcf7b520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:38.124617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffefef5-0355-42bd-9d01-2c3bbcf7b520 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:38.125062 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffefef5-0355-42bd-9d01-2c3bbcf7b520 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:38.125583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffefef5-0355-42bd-9d01-2c3bbcf7b520 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:38.127310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffefef5-0355-42bd-9d01-2c3bbcf7b520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:38.129045 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffefef5-0355-42bd-9d01-2c3bbcf7b520 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:38.148589 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1368/2744] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:38.150416 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffefef5-0355-42bd-9d01-2c3bbcf7b520 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:38.169250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffefef5-0355-42bd-9d01-2c3bbcf7b520 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:38.186151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5527e97-0edc-4547-8a88-7666587068c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:38.186151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5527e97-0edc-4547-8a88-7666587068c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:38.186151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5527e97-0edc-4547-8a88-7666587068c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:38.186151 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5527e97-0edc-4547-8a88-7666587068c6 admin admin] PATH_INFO: `/v3/projects/c9a98126fca5457999d2fbaa0fb66cbb/users/d01db669717841059bd13216750e5597/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:38.186634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5527e97-0edc-4547-8a88-7666587068c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a98126fca5457999d2fbaa0fb66cbb, user_id=d01db669717841059bd13216750e5597, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:38.190028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5527e97-0edc-4547-8a88-7666587068c6 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:38.220548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1369/2745] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:38 2022] PUT /identity/v3/projects/c9a98126fca5457999d2fbaa0fb66cbb/users/d01db669717841059bd13216750e5597/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:38.234908 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a758d-bfbd-4d37-9945-3d6d9a21e131 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:38.235200 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a758d-bfbd-4d37-9945-3d6d9a21e131 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:38.235482 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a758d-bfbd-4d37-9945-3d6d9a21e131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:38.256904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e00530-9f15-4c8b-a1c6-7dd03990ad4f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:38.288868 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-660a758d-bfbd-4d37-9945-3d6d9a21e131 None None] MFA Rules not processed for user `d01db669717841059bd13216750e5597`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:38.290718 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-660a758d-bfbd-4d37-9945-3d6d9a21e131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:38.348933 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:38.350278 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:38.350856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:38.351315 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] PATH_INFO: `/v3/projects/0de3dbd984334a9e9842c8756ef37a9b` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:38.352854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de3dbd984334a9e9842c8756ef37a9b)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:38.355745 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:38.421278 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1370/2746] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:38.434669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a758d-bfbd-4d37-9945-3d6d9a21e131 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:38.434669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-660a758d-bfbd-4d37-9945-3d6d9a21e131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:38.496324 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a758d-bfbd-4d37-9945-3d6d9a21e131 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:38.505218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de3dbd984334a9e9842c8756ef37a9b', 'request_id': 'req-8a53d174-9cac-4c20-a325-28d448bcd599'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:38.513371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de3dbd984334a9e9842c8756ef37a9b', 'request_id': 'req-8a53d174-9cac-4c20-a325-28d448bcd599'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:38.520243 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de3dbd984334a9e9842c8756ef37a9b', 'request_id': 'req-8a53d174-9cac-4c20-a325-28d448bcd599'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:38.548419 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964e176-1589-448c-8dce-cde76958d81d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:38.549068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964e176-1589-448c-8dce-cde76958d81d service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:38.549346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964e176-1589-448c-8dce-cde76958d81d service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:38.549624 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964e176-1589-448c-8dce-cde76958d81d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:38.592812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] The token cache is being invalidate because project 0de3dbd984334a9e9842c8756ef37a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:38.593301 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:38.593992 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-8a53d174-9cac-4c20-a325-28d448bcd599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:38.596045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1377/2747] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:38 2022] DELETE /identity/v3/projects/0de3dbd984334a9e9842c8756ef37a9b => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:38.600845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92932e7b-8c0c-4ec3-9ed8-0acc56d4ed85 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:38.601240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92932e7b-8c0c-4ec3-9ed8-0acc56d4ed85 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:38.601620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92932e7b-8c0c-4ec3-9ed8-0acc56d4ed85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:38.615292 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964e176-1589-448c-8dce-cde76958d81d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:38.616988 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964e176-1589-448c-8dce-cde76958d81d service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:38.622349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-92932e7b-8c0c-4ec3-9ed8-0acc56d4ed85 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:38.624316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-92932e7b-8c0c-4ec3-9ed8-0acc56d4ed85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:38.627793 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1371/2748] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:38.691218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1378/2749] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:38.700212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dde3e-0328-4bde-b50f-4895a38188ac None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:38.700507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dde3e-0328-4bde-b50f-4895a38188ac None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:38.700783 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dde3e-0328-4bde-b50f-4895a38188ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:38.724214 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-707dde3e-0328-4bde-b50f-4895a38188ac None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:38.730343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-707dde3e-0328-4bde-b50f-4895a38188ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:38.738786 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1372/2750] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:38.743612 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92932e7b-8c0c-4ec3-9ed8-0acc56d4ed85 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:38.754532 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92932e7b-8c0c-4ec3-9ed8-0acc56d4ed85 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:38.760861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab9e2d-077d-4bb7-acfb-0bb4385f2856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:38.761448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9e2d-077d-4bb7-acfb-0bb4385f2856 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:38.761756 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9e2d-077d-4bb7-acfb-0bb4385f2856 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:38.762024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9e2d-077d-4bb7-acfb-0bb4385f2856 admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:38.762817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab9e2d-077d-4bb7-acfb-0bb4385f2856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:38.763126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab9e2d-077d-4bb7-acfb-0bb4385f2856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:38.765445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab9e2d-077d-4bb7-acfb-0bb4385f2856 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:38.776893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:38.780856 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1379/2751] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:38.785487 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dde3e-0328-4bde-b50f-4895a38188ac None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:38.798192 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dde3e-0328-4bde-b50f-4895a38188ac None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:38.803618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856580b-b183-4ac3-b3f4-4861dc5cb066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:38.804363 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856580b-b183-4ac3-b3f4-4861dc5cb066 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:38.804615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856580b-b183-4ac3-b3f4-4861dc5cb066 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:38.804830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856580b-b183-4ac3-b3f4-4861dc5cb066 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:38.806177 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856580b-b183-4ac3-b3f4-4861dc5cb066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:38.808144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856580b-b183-4ac3-b3f4-4861dc5cb066 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:38.860670 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab9e2d-077d-4bb7-acfb-0bb4385f2856 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:38.862444 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6eab9e2d-077d-4bb7-acfb-0bb4385f2856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:38.876195 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab9e2d-077d-4bb7-acfb-0bb4385f2856 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:38.880163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbc5e1-61d0-4c1d-8224-350cafb421cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:38.880962 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbc5e1-61d0-4c1d-8224-350cafb421cd service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:38.881248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbc5e1-61d0-4c1d-8224-350cafb421cd service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:38.881527 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbc5e1-61d0-4c1d-8224-350cafb421cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:38.883417 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-84dbc5e1-61d0-4c1d-8224-350cafb421cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:38.928209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1373/2752] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:38 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:38.928959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856580b-b183-4ac3-b3f4-4861dc5cb066 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:38.931138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2856580b-b183-4ac3-b3f4-4861dc5cb066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:38.975569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbc5e1-61d0-4c1d-8224-350cafb421cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:38.977657 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbc5e1-61d0-4c1d-8224-350cafb421cd service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:39.003341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1380/2753] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:39.004925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbc5e1-61d0-4c1d-8224-350cafb421cd service cinder] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:39.012070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856580b-b183-4ac3-b3f4-4861dc5cb066 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:39.025433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:39.025433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.025433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.025433 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] PATH_INFO: `/v3/users/6b92ec17aad046c489c0c7c7c0318a6f` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.027028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b92ec17aad046c489c0c7c7c0318a6f)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:39.027028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:39.029871 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbc5e1-61d0-4c1d-8224-350cafb421cd service cinder] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:39.044870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:39.045649 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee2737-4910-4e5a-80f2-204d021ef6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:39.046360 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee2737-4910-4e5a-80f2-204d021ef6db admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.046699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee2737-4910-4e5a-80f2-204d021ef6db admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.053568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee2737-4910-4e5a-80f2-204d021ef6db admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.053568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee2737-4910-4e5a-80f2-204d021ef6db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:39.053568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee2737-4910-4e5a-80f2-204d021ef6db admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:39.174996 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1381/2754] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:39 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:39.179155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee2737-4910-4e5a-80f2-204d021ef6db admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:39.197512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee2737-4910-4e5a-80f2-204d021ef6db admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:39.203575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cb839-b0a6-442a-837b-99dde95da673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:39.204797 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cb839-b0a6-442a-837b-99dde95da673 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.205268 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cb839-b0a6-442a-837b-99dde95da673 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.205693 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cb839-b0a6-442a-837b-99dde95da673 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.206785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cb839-b0a6-442a-837b-99dde95da673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:39.208879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839cb839-b0a6-442a-837b-99dde95da673 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:39.224460 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1382/2755] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:39.229216 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cb839-b0a6-442a-837b-99dde95da673 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:39.241903 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cb839-b0a6-442a-837b-99dde95da673 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:39.247742 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa7fb7-2efe-41f5-86fa-2377f279afbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:39.253294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa7fb7-2efe-41f5-86fa-2377f279afbd admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.253294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa7fb7-2efe-41f5-86fa-2377f279afbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.253294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa7fb7-2efe-41f5-86fa-2377f279afbd admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.253294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa7fb7-2efe-41f5-86fa-2377f279afbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:39.253294 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa7fb7-2efe-41f5-86fa-2377f279afbd admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:39.270034 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b92ec17aad046c489c0c7c7c0318a6f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:39.279963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1383/2756] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:39.280857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa7fb7-2efe-41f5-86fa-2377f279afbd admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:39.287128 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b92ec17aad046c489c0c7c7c0318a6f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:39.300609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa7fb7-2efe-41f5-86fa-2377f279afbd admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:39.306724 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560a0f-5c60-4839-a570-5badd90f7478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:39.307641 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560a0f-5c60-4839-a570-5badd90f7478 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.308167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560a0f-5c60-4839-a570-5badd90f7478 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.308167 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560a0f-5c60-4839-a570-5badd90f7478 admin admin] PATH_INFO: `/v3/projects/261bd30389f8434486e743be7ffd6688/users/a40b22d7306543538e1ffc5677ba8515/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.331531 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560a0f-5c60-4839-a570-5badd90f7478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=261bd30389f8434486e743be7ffd6688, user_id=a40b22d7306543538e1ffc5677ba8515, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:39.335503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560a0f-5c60-4839-a570-5badd90f7478 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:39.398101 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b92ec17aad046c489c0c7c7c0318a6f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:39.398957 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b92ec17aad046c489c0c7c7c0318a6f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:39.399627 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-3a1118d5-46ec-4964-8e79-c501facb0b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b92ec17aad046c489c0c7c7c0318a6f'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:39.425008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1374/2757] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:38 2022] DELETE /identity/v3/users/6b92ec17aad046c489c0c7c7c0318a6f => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:39.431800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de4efc-a076-4b16-83b2-bdb4f5ea9e1f None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.435146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de4efc-a076-4b16-83b2-bdb4f5ea9e1f None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.435146 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de4efc-a076-4b16-83b2-bdb4f5ea9e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.437193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1384/2758] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:39 2022] PUT /identity/v3/projects/261bd30389f8434486e743be7ffd6688/users/a40b22d7306543538e1ffc5677ba8515/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:39.448618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72241c-5f0b-4d91-86e0-b04222acbf46 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.448886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72241c-5f0b-4d91-86e0-b04222acbf46 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.449160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72241c-5f0b-4d91-86e0-b04222acbf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.472353 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-81de4efc-a076-4b16-83b2-bdb4f5ea9e1f None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:39.473697 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-81de4efc-a076-4b16-83b2-bdb4f5ea9e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:39.488219 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-3c72241c-5f0b-4d91-86e0-b04222acbf46 None None] MFA Rules not processed for user `a40b22d7306543538e1ffc5677ba8515`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:39.499371 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3c72241c-5f0b-4d91-86e0-b04222acbf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:39.592959 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1375/2759] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:39.610941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de4efc-a076-4b16-83b2-bdb4f5ea9e1f None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:39.612398 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-81de4efc-a076-4b16-83b2-bdb4f5ea9e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:39.618767 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1385/2760] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:39.621863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72241c-5f0b-4d91-86e0-b04222acbf46 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:39.639905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3c72241c-5f0b-4d91-86e0-b04222acbf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:39.684129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:39.692590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72241c-5f0b-4d91-86e0-b04222acbf46 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:39.702290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc92e74-3e68-4b4a-8c2a-b9f3ad12babe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:39.702290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc92e74-3e68-4b4a-8c2a-b9f3ad12babe admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.702290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc92e74-3e68-4b4a-8c2a-b9f3ad12babe admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.702290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc92e74-3e68-4b4a-8c2a-b9f3ad12babe admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.703063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc92e74-3e68-4b4a-8c2a-b9f3ad12babe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:39.705064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc92e74-3e68-4b4a-8c2a-b9f3ad12babe admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:39.766527 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de4efc-a076-4b16-83b2-bdb4f5ea9e1f None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:39.779622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e8704-67d0-4b6c-a27a-bb24fdb5c41a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:39.781827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e8704-67d0-4b6c-a27a-bb24fdb5c41a service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.782110 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e8704-67d0-4b6c-a27a-bb24fdb5c41a service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.783248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e8704-67d0-4b6c-a27a-bb24fdb5c41a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.817531 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e8704-67d0-4b6c-a27a-bb24fdb5c41a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:39.821925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e8704-67d0-4b6c-a27a-bb24fdb5c41a service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:39.838201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1376/2761] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:39.911633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1386/2762] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:39 2022] POST /identity/v3/projects => generated 383 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:39.915977 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e8704-67d0-4b6c-a27a-bb24fdb5c41a service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:39.941171 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e8704-67d0-4b6c-a27a-bb24fdb5c41a service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.046424 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913a989-2115-4f74-90f5-72ed93541711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.047105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913a989-2115-4f74-90f5-72ed93541711 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.047494 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913a989-2115-4f74-90f5-72ed93541711 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.047875 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913a989-2115-4f74-90f5-72ed93541711 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.048739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913a989-2115-4f74-90f5-72ed93541711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.050878 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913a989-2115-4f74-90f5-72ed93541711 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.147787 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1377/2763] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:39 2022] POST /identity/v3/users => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:40.152399 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc92e74-3e68-4b4a-8c2a-b9f3ad12babe admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:40.167541 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc92e74-3e68-4b4a-8c2a-b9f3ad12babe admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.173162 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4b603-4e46-4040-9af3-42df71286895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.174071 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4b603-4e46-4040-9af3-42df71286895 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.174594 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4b603-4e46-4040-9af3-42df71286895 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.175052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4b603-4e46-4040-9af3-42df71286895 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.175896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4b603-4e46-4040-9af3-42df71286895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.177686 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4b603-4e46-4040-9af3-42df71286895 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.193765 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1387/2764] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:40.194851 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913a989-2115-4f74-90f5-72ed93541711 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:40.211435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913a989-2115-4f74-90f5-72ed93541711 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.215856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:40.219197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4a137-58ea-4c95-bde8-41a917fb2e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.220198 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4a137-58ea-4c95-bde8-41a917fb2e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.220608 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4a137-58ea-4c95-bde8-41a917fb2e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.220986 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4a137-58ea-4c95-bde8-41a917fb2e5c admin admin] PATH_INFO: `/v3/projects/817282b3dff74ed8a02ed25101f21b70/users/3b0d7bdfb47b4251a7c4b5da25b20a0b/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.256347 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4a137-58ea-4c95-bde8-41a917fb2e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=817282b3dff74ed8a02ed25101f21b70, user_id=3b0d7bdfb47b4251a7c4b5da25b20a0b, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.261383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4a137-58ea-4c95-bde8-41a917fb2e5c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.331857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1378/2765] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:40 2022] PUT /identity/v3/projects/817282b3dff74ed8a02ed25101f21b70/users/3b0d7bdfb47b4251a7c4b5da25b20a0b/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:40.335488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4b603-4e46-4040-9af3-42df71286895 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:40.345850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4b603-4e46-4040-9af3-42df71286895 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.412643 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f85c0-e595-42c2-8f61-f08a5419b34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.413325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f85c0-e595-42c2-8f61-f08a5419b34c admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.413769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f85c0-e595-42c2-8f61-f08a5419b34c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.414056 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f85c0-e595-42c2-8f61-f08a5419b34c admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.415392 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f85c0-e595-42c2-8f61-f08a5419b34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.417425 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f85c0-e595-42c2-8f61-f08a5419b34c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.427956 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1388/2766] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:40.432792 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4a137-58ea-4c95-bde8-41a917fb2e5c admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:40.443999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4a137-58ea-4c95-bde8-41a917fb2e5c admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.450064 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d76b2d-3dc8-4f40-9623-f2c25f5b4515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.451246 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d76b2d-3dc8-4f40-9623-f2c25f5b4515 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.451958 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d76b2d-3dc8-4f40-9623-f2c25f5b4515 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.452134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d76b2d-3dc8-4f40-9623-f2c25f5b4515 admin admin] PATH_INFO: `/v3/projects/817282b3dff74ed8a02ed25101f21b70/users/3b0d7bdfb47b4251a7c4b5da25b20a0b/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.456615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d76b2d-3dc8-4f40-9623-f2c25f5b4515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=817282b3dff74ed8a02ed25101f21b70, user_id=3b0d7bdfb47b4251a7c4b5da25b20a0b, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.459764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d76b2d-3dc8-4f40-9623-f2c25f5b4515 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.473453 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1379/2767] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:40 2022] PUT /identity/v3/projects/817282b3dff74ed8a02ed25101f21b70/users/3b0d7bdfb47b4251a7c4b5da25b20a0b/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:40.544618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f2fbb-d813-4b9d-9518-ab0e607b3e22 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.544911 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f2fbb-d813-4b9d-9518-ab0e607b3e22 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.545194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f2fbb-d813-4b9d-9518-ab0e607b3e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.575218 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-560f2fbb-d813-4b9d-9518-ab0e607b3e22 None None] MFA Rules not processed for user `3b0d7bdfb47b4251a7c4b5da25b20a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:40.576479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-560f2fbb-d813-4b9d-9518-ab0e607b3e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:40.649656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1389/2768] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:40.662095 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d76b2d-3dc8-4f40-9623-f2c25f5b4515 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:40.664214 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-25d76b2d-3dc8-4f40-9623-f2c25f5b4515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:40.677964 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f2fbb-d813-4b9d-9518-ab0e607b3e22 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:40.681238 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-560f2fbb-d813-4b9d-9518-ab0e607b3e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:40.756079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:40.762011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d76b2d-3dc8-4f40-9623-f2c25f5b4515 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.767395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f7ab0-f08d-4dc0-a37a-230c1a868ef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.768755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f7ab0-f08d-4dc0-a37a-230c1a868ef2 service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.769122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f7ab0-f08d-4dc0-a37a-230c1a868ef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.769241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f2fbb-d813-4b9d-9518-ab0e607b3e22 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.769543 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f7ab0-f08d-4dc0-a37a-230c1a868ef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.775925 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.777155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.777608 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.778042 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] PATH_INFO: `/v3/projects/7596aa36b16d45a0957a306d19a56113` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.780667 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7596aa36b16d45a0957a306d19a56113)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.783795 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.785493 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f7ab0-f08d-4dc0-a37a-230c1a868ef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.788341 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f7ab0-f08d-4dc0-a37a-230c1a868ef2 service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.822106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1380/2769] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:40.947279 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7596aa36b16d45a0957a306d19a56113', 'request_id': 'req-820deea6-3f52-4c75-8bbe-304df5045b19'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.953556 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7596aa36b16d45a0957a306d19a56113', 'request_id': 'req-820deea6-3f52-4c75-8bbe-304df5045b19'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.959713 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7596aa36b16d45a0957a306d19a56113', 'request_id': 'req-820deea6-3f52-4c75-8bbe-304df5045b19'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:41.050841 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f7ab0-f08d-4dc0-a37a-230c1a868ef2 service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.056092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-545f7ab0-f08d-4dc0-a37a-230c1a868ef2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:41.071617 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f7ab0-f08d-4dc0-a37a-230c1a868ef2 service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.095894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] The token cache is being invalidate because project 7596aa36b16d45a0957a306d19a56113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:41.096177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:41.096944 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-820deea6-3f52-4c75-8bbe-304df5045b19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:41.102866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1390/2770] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:40 2022] DELETE /identity/v3/projects/7596aa36b16d45a0957a306d19a56113 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:41.104529 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8be18-b5bf-4b59-8459-28757778289b None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.104824 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8be18-b5bf-4b59-8459-28757778289b None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.105109 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8be18-b5bf-4b59-8459-28757778289b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.120351 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:41.148140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-88d8be18-b5bf-4b59-8459-28757778289b None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:41.149945 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-88d8be18-b5bf-4b59-8459-28757778289b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:41.197374 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb5962-4fba-4611-a5e5-e6809c93b06b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.199134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5962-4fba-4611-a5e5-e6809c93b06b service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.200190 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5962-4fba-4611-a5e5-e6809c93b06b service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.201860 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5962-4fba-4611-a5e5-e6809c93b06b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.206105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-cabb5962-4fba-4611-a5e5-e6809c93b06b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:41.280562 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1391/2771] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:41.289712 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0237a3-c619-4a73-a105-0b8adc576d95 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.289986 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0237a3-c619-4a73-a105-0b8adc576d95 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.290489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0237a3-c619-4a73-a105-0b8adc576d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.315499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-1b0237a3-c619-4a73-a105-0b8adc576d95 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:41.316892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1b0237a3-c619-4a73-a105-0b8adc576d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:41.327098 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1392/2772] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:41.331479 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0237a3-c619-4a73-a105-0b8adc576d95 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.336117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb5962-4fba-4611-a5e5-e6809c93b06b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.338250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb5962-4fba-4611-a5e5-e6809c93b06b service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.345588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0237a3-c619-4a73-a105-0b8adc576d95 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.350985 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446a353-02ff-49fa-be7c-d3d767b8faad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.351810 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446a353-02ff-49fa-be7c-d3d767b8faad admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.352385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1381/2773] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:41.352518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446a353-02ff-49fa-be7c-d3d767b8faad admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.352606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446a353-02ff-49fa-be7c-d3d767b8faad admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.353518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446a353-02ff-49fa-be7c-d3d767b8faad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:41.353893 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446a353-02ff-49fa-be7c-d3d767b8faad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.355589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446a353-02ff-49fa-be7c-d3d767b8faad admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.398277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1393/2774] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:41.414990 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb5962-4fba-4611-a5e5-e6809c93b06b service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.428488 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb5962-4fba-4611-a5e5-e6809c93b06b service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.434197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c379a2-73fd-4503-8f26-de1eb97f86ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.439873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c379a2-73fd-4503-8f26-de1eb97f86ff admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.439873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c379a2-73fd-4503-8f26-de1eb97f86ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.439873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c379a2-73fd-4503-8f26-de1eb97f86ff admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.439873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c379a2-73fd-4503-8f26-de1eb97f86ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.439873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c379a2-73fd-4503-8f26-de1eb97f86ff admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.543341 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1382/2775] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:41 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:41.548703 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446a353-02ff-49fa-be7c-d3d767b8faad admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.561178 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446a353-02ff-49fa-be7c-d3d767b8faad admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.643055 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19df482-460e-4f3c-9a87-1b9431dc12c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.643815 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19df482-460e-4f3c-9a87-1b9431dc12c7 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.644242 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19df482-460e-4f3c-9a87-1b9431dc12c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.644929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19df482-460e-4f3c-9a87-1b9431dc12c7 admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.645400 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19df482-460e-4f3c-9a87-1b9431dc12c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.648114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19df482-460e-4f3c-9a87-1b9431dc12c7 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.742246 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1394/2776] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:41 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:41.746993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c379a2-73fd-4503-8f26-de1eb97f86ff admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.777795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c379a2-73fd-4503-8f26-de1eb97f86ff admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.788571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74bcd3-399f-43ab-bcfd-1050c14a2c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.791234 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74bcd3-399f-43ab-bcfd-1050c14a2c11 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.797606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74bcd3-399f-43ab-bcfd-1050c14a2c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.797606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74bcd3-399f-43ab-bcfd-1050c14a2c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.797606 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74bcd3-399f-43ab-bcfd-1050c14a2c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.805686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74bcd3-399f-43ab-bcfd-1050c14a2c11 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.821187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1383/2777] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:41.829432 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19df482-460e-4f3c-9a87-1b9431dc12c7 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.842298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19df482-460e-4f3c-9a87-1b9431dc12c7 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.856128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8f9b4-e322-41ef-841d-9f2de605cbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.857644 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f9b4-e322-41ef-841d-9f2de605cbec admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.858110 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f9b4-e322-41ef-841d-9f2de605cbec admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.859339 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f9b4-e322-41ef-841d-9f2de605cbec admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.873030 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8f9b4-e322-41ef-841d-9f2de605cbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.875257 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8f9b4-e322-41ef-841d-9f2de605cbec admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.899537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1395/2778] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:41.905258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74bcd3-399f-43ab-bcfd-1050c14a2c11 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.939691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74bcd3-399f-43ab-bcfd-1050c14a2c11 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.949311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac0cbe-1af2-4a45-bad8-883db1fef0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.950379 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac0cbe-1af2-4a45-bad8-883db1fef0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.950885 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac0cbe-1af2-4a45-bad8-883db1fef0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.951267 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac0cbe-1af2-4a45-bad8-883db1fef0a7 admin admin] PATH_INFO: `/v3/projects/9918a1f8a1204a76aea5a3c14ff45e30/users/ed3c7982a4ea4d0aafb238eadc856094/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.978427 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac0cbe-1af2-4a45-bad8-883db1fef0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9918a1f8a1204a76aea5a3c14ff45e30, user_id=ed3c7982a4ea4d0aafb238eadc856094, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.983539 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac0cbe-1af2-4a45-bad8-883db1fef0a7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.994915 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:42.021425 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:42.081431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1384/2779] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:41 2022] PUT /identity/v3/projects/9918a1f8a1204a76aea5a3c14ff45e30/users/ed3c7982a4ea4d0aafb238eadc856094/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:42.083556 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17817e25-7028-4637-adc6-2fc93b125481 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.084020 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17817e25-7028-4637-adc6-2fc93b125481 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.084518 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17817e25-7028-4637-adc6-2fc93b125481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.125820 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac0cbe-1af2-4a45-bad8-883db1fef0a7 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:42.127686 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-76ac0cbe-1af2-4a45-bad8-883db1fef0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.136838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-17817e25-7028-4637-adc6-2fc93b125481 None None] MFA Rules not processed for user `ed3c7982a4ea4d0aafb238eadc856094`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:42.140045 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-17817e25-7028-4637-adc6-2fc93b125481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.244215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:42.256690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac0cbe-1af2-4a45-bad8-883db1fef0a7 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:42.262038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:42.263103 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.263580 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.264026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] PATH_INFO: `/v3/users/a40b22d7306543538e1ffc5677ba8515` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.265535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a40b22d7306543538e1ffc5677ba8515)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:42.268750 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:42.286273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1396/2780] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:42.301214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17817e25-7028-4637-adc6-2fc93b125481 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:42.303173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-17817e25-7028-4637-adc6-2fc93b125481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.353253 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:42.365335 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17817e25-7028-4637-adc6-2fc93b125481 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:42.369689 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd6a60-76eb-4d39-b220-a3a3de713263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:42.370347 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd6a60-76eb-4d39-b220-a3a3de713263 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.370668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd6a60-76eb-4d39-b220-a3a3de713263 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.370946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd6a60-76eb-4d39-b220-a3a3de713263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.380060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd6a60-76eb-4d39-b220-a3a3de713263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:42.381512 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd6a60-76eb-4d39-b220-a3a3de713263 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:42.391566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1397/2781] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:42.392860 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530d9db-1422-42d1-a115-cac7ed20160c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.393146 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530d9db-1422-42d1-a115-cac7ed20160c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.393422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530d9db-1422-42d1-a115-cac7ed20160c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.424237 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a530d9db-1422-42d1-a115-cac7ed20160c None None] MFA Rules not processed for user `a7d69a23418e491f883936f409067f26`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:42.425210 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a530d9db-1422-42d1-a115-cac7ed20160c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.472384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a40b22d7306543538e1ffc5677ba8515'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.483312 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a40b22d7306543538e1ffc5677ba8515'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.502766 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1398/2782] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:42.503627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530d9db-1422-42d1-a115-cac7ed20160c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:42.519286 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530d9db-1422-42d1-a115-cac7ed20160c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:42.524318 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:42.526791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.526791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.526791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] PATH_INFO: `/v3/users/ed3c7982a4ea4d0aafb238eadc856094` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.527090 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed3c7982a4ea4d0aafb238eadc856094)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:42.528913 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:42.575071 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a40b22d7306543538e1ffc5677ba8515'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.575691 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a40b22d7306543538e1ffc5677ba8515'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.576314 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a40b22d7306543538e1ffc5677ba8515'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.594967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1385/2783] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:42 2022] DELETE /identity/v3/users/a40b22d7306543538e1ffc5677ba8515 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:42.595895 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:42.597227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.655291 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e36a4d-d1b1-4a52-a876-abd69b953812 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:42.658932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877892a-8d79-4012-8529-9d8beae2aecd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e12361da5f849ee97607ca68ee6ef10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:42.659618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877892a-8d79-4012-8529-9d8beae2aecd service glance] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.659887 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877892a-8d79-4012-8529-9d8beae2aecd service glance] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.660163 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877892a-8d79-4012-8529-9d8beae2aecd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.661941 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9877892a-8d79-4012-8529-9d8beae2aecd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.718349 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877892a-8d79-4012-8529-9d8beae2aecd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:42.719829 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877892a-8d79-4012-8529-9d8beae2aecd service glance] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:42.729114 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1386/2784] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:42.730344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6621f-c215-4835-89dd-8d33cb205f54 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.730663 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6621f-c215-4835-89dd-8d33cb205f54 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.730944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6621f-c215-4835-89dd-8d33cb205f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.748060 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-a0d6621f-c215-4835-89dd-8d33cb205f54 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:42.749261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a0d6621f-c215-4835-89dd-8d33cb205f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.805825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed3c7982a4ea4d0aafb238eadc856094'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.806896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1387/2785] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:42.813895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed3c7982a4ea4d0aafb238eadc856094'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.814622 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed3c7982a4ea4d0aafb238eadc856094'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.818998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6621f-c215-4835-89dd-8d33cb205f54 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:42.821188 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-a0d6621f-c215-4835-89dd-8d33cb205f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.858733 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:42.882250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6621f-c215-4835-89dd-8d33cb205f54 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:42.886588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab9f83-e4c7-477b-b806-2b663e4a79f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:42.887253 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab9f83-e4c7-477b-b806-2b663e4a79f6 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.887537 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab9f83-e4c7-477b-b806-2b663e4a79f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.887864 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab9f83-e4c7-477b-b806-2b663e4a79f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.900557 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab9f83-e4c7-477b-b806-2b663e4a79f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:42.905345 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab9f83-e4c7-477b-b806-2b663e4a79f6 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:42.914772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1388/2786] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:42.932120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed3c7982a4ea4d0aafb238eadc856094'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.933229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed3c7982a4ea4d0aafb238eadc856094'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.953207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1399/2787] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:42 2022] DELETE /identity/v3/users/ed3c7982a4ea4d0aafb238eadc856094 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:42.957920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076d2d2-a5f7-4112-8430-2edb6c03ffeb None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.958286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076d2d2-a5f7-4112-8430-2edb6c03ffeb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.958742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076d2d2-a5f7-4112-8430-2edb6c03ffeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.977719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e076d2d2-a5f7-4112-8430-2edb6c03ffeb None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:42.983812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e076d2d2-a5f7-4112-8430-2edb6c03ffeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:43.041835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1389/2788] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:43.053877 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:43.056022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:43.079471 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:43.081475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db1271-2d38-4c07-ad4e-682d6d64cdba None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:43.081475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db1271-2d38-4c07-ad4e-682d6d64cdba None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:43.081475 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db1271-2d38-4c07-ad4e-682d6d64cdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:43.111310 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e8db1271-2d38-4c07-ad4e-682d6d64cdba None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:43.112656 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e8db1271-2d38-4c07-ad4e-682d6d64cdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:43.123718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce679c-f7f8-4e19-b84e-f28b23a14a08 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:43.127489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e2351-6d42-4911-8a29-d08ff7540bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:43.128148 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e2351-6d42-4911-8a29-d08ff7540bec service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:43.132888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e2351-6d42-4911-8a29-d08ff7540bec service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:43.132888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e2351-6d42-4911-8a29-d08ff7540bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:43.141409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e2351-6d42-4911-8a29-d08ff7540bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:43.143061 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e2351-6d42-4911-8a29-d08ff7540bec service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:43.153814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1400/2789] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:43.198696 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1390/2790] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 06:06:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:43.329325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e2351-6d42-4911-8a29-d08ff7540bec service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:43.330954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-5d7e2351-6d42-4911-8a29-d08ff7540bec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:43.387050 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=100 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:43.412186 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e2351-6d42-4911-8a29-d08ff7540bec service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:43.418096 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1a97d-35a8-4df9-8aae-c583273667a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:43.418814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1a97d-35a8-4df9-8aae-c583273667a2 service swift] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:43.419362 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1a97d-35a8-4df9-8aae-c583273667a2 service swift] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:43.420762 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1a97d-35a8-4df9-8aae-c583273667a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:43.428872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1a97d-35a8-4df9-8aae-c583273667a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:43.430569 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1a97d-35a8-4df9-8aae-c583273667a2 service swift] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:43.439365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1401/2791] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:06:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:06:43.459731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db1271-2d38-4c07-ad4e-682d6d64cdba None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:43.459731 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e8db1271-2d38-4c07-ad4e-682d6d64cdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:43.487817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db1271-2d38-4c07-ad4e-682d6d64cdba None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:43.488566 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:43.489243 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:43.489602 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:43.489945 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] PATH_INFO: `/v3/projects/261bd30389f8434486e743be7ffd6688` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:43.491664 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=261bd30389f8434486e743be7ffd6688)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:43.494712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:43.661461 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '261bd30389f8434486e743be7ffd6688', 'request_id': 'req-32218121-0dfb-4416-a3df-1619051d9a84'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.685437 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '261bd30389f8434486e743be7ffd6688', 'request_id': 'req-32218121-0dfb-4416-a3df-1619051d9a84'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.725985 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '261bd30389f8434486e743be7ffd6688', 'request_id': 'req-32218121-0dfb-4416-a3df-1619051d9a84'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.726187 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1a97d-35a8-4df9-8aae-c583273667a2 service swift] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:43.727476 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-ccb1a97d-35a8-4df9-8aae-c583273667a2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:43.878216 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] The token cache is being invalidate because project 261bd30389f8434486e743be7ffd6688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:43.878929 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.882263 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.886083 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1391/2792] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:43 2022] DELETE /identity/v3/projects/261bd30389f8434486e743be7ffd6688 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:43.887441 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:43.889803 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:43.942014 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1a97d-35a8-4df9-8aae-c583273667a2 service swift] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:43.988010 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=100 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:44.029597 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32218121-0dfb-4416-a3df-1619051d9a84 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:44.044245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:44.045162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.045447 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.045727 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] PATH_INFO: `/v3/projects/9918a1f8a1204a76aea5a3c14ff45e30` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.047059 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] RBAC: Authorizing `identity:delete_project(project_id=9918a1f8a1204a76aea5a3c14ff45e30)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:44.049066 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:44.056895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da701eb-246f-46d1-9507-ac3e9110d9c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:44.058091 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da701eb-246f-46d1-9507-ac3e9110d9c3 service placement] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.058588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da701eb-246f-46d1-9507-ac3e9110d9c3 service placement] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.059183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da701eb-246f-46d1-9507-ac3e9110d9c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.061805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2da701eb-246f-46d1-9507-ac3e9110d9c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:44.076661 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da701eb-246f-46d1-9507-ac3e9110d9c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:44.079225 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da701eb-246f-46d1-9507-ac3e9110d9c3 service placement] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:44.094420 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1402/2793] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:44.095457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da701eb-246f-46d1-9507-ac3e9110d9c3 service placement] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:44.096812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-2da701eb-246f-46d1-9507-ac3e9110d9c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:44.119433 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da701eb-246f-46d1-9507-ac3e9110d9c3 service placement] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:44.126491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:44.127707 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.127908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.128191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] PATH_INFO: `/v3/users/3b0d7bdfb47b4251a7c4b5da25b20a0b` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.129445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b0d7bdfb47b4251a7c4b5da25b20a0b)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:44.131560 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:44.226184 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9918a1f8a1204a76aea5a3c14ff45e30', 'request_id': 'req-645ba167-2567-4582-931f-7cbadea3faba'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.236835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9918a1f8a1204a76aea5a3c14ff45e30', 'request_id': 'req-645ba167-2567-4582-931f-7cbadea3faba'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.244435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9918a1f8a1204a76aea5a3c14ff45e30', 'request_id': 'req-645ba167-2567-4582-931f-7cbadea3faba'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.252415 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:44.347227 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b0d7bdfb47b4251a7c4b5da25b20a0b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.361720 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b0d7bdfb47b4251a7c4b5da25b20a0b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.362685 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b0d7bdfb47b4251a7c4b5da25b20a0b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.367277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] The token cache is being invalidate because project 9918a1f8a1204a76aea5a3c14ff45e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:44.367856 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.368936 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-645ba167-2567-4582-931f-7cbadea3faba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.373507 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1392/2794] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:43 2022] DELETE /identity/v3/projects/9918a1f8a1204a76aea5a3c14ff45e30 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:44.378925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f67f9b-1fe4-4803-bf55-3f6b33097601 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.379333 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f67f9b-1fe4-4803-bf55-3f6b33097601 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.379708 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f67f9b-1fe4-4803-bf55-3f6b33097601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.412120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-72f67f9b-1fe4-4803-bf55-3f6b33097601 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:44.413920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-72f67f9b-1fe4-4803-bf55-3f6b33097601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:44.475453 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b0d7bdfb47b4251a7c4b5da25b20a0b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.479747 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-1ad3ddc2-5cc7-4f8e-9ccb-01b82ebf2f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b0d7bdfb47b4251a7c4b5da25b20a0b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.505224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1403/2795] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:44 2022] DELETE /identity/v3/users/3b0d7bdfb47b4251a7c4b5da25b20a0b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:44.553181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1393/2796] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:44.567607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c3364-7dbf-440b-8710-a4849c3fea61 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.568093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c3364-7dbf-440b-8710-a4849c3fea61 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.568700 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c3364-7dbf-440b-8710-a4849c3fea61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.602375 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-a28c3364-7dbf-440b-8710-a4849c3fea61 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:44.604599 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-a28c3364-7dbf-440b-8710-a4849c3fea61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:44.843627 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1404/2797] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:44.846460 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f67f9b-1fe4-4803-bf55-3f6b33097601 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:44.863106 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f67f9b-1fe4-4803-bf55-3f6b33097601 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:44.872621 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d3695-097a-4a31-9c52-817175ee1044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:44.876070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d3695-097a-4a31-9c52-817175ee1044 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.876753 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d3695-097a-4a31-9c52-817175ee1044 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.877338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d3695-097a-4a31-9c52-817175ee1044 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.878880 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d3695-097a-4a31-9c52-817175ee1044 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:44.879579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d3695-097a-4a31-9c52-817175ee1044 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:44.882382 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d3695-097a-4a31-9c52-817175ee1044 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:44.898378 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1394/2798] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:44.903909 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c3364-7dbf-440b-8710-a4849c3fea61 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:44.924606 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c3364-7dbf-440b-8710-a4849c3fea61 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:44.932248 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfcc6e-8040-42c6-91f8-15e079f0739b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:44.933150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfcc6e-8040-42c6-91f8-15e079f0739b admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.933551 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfcc6e-8040-42c6-91f8-15e079f0739b admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.933820 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfcc6e-8040-42c6-91f8-15e079f0739b admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.934609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfcc6e-8040-42c6-91f8-15e079f0739b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:44.937348 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfcc6e-8040-42c6-91f8-15e079f0739b admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.057428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1405/2799] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:44 2022] POST /identity/v3/projects => generated 397 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:45.061734 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d3695-097a-4a31-9c52-817175ee1044 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.078002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d3695-097a-4a31-9c52-817175ee1044 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.191133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c356d-9dfc-4df5-96d4-e8cc3b25de5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.191952 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c356d-9dfc-4df5-96d4-e8cc3b25de5f admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.192392 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c356d-9dfc-4df5-96d4-e8cc3b25de5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.192822 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c356d-9dfc-4df5-96d4-e8cc3b25de5f admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.193648 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c356d-9dfc-4df5-96d4-e8cc3b25de5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.196009 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c356d-9dfc-4df5-96d4-e8cc3b25de5f admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.228872 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfcc6e-8040-42c6-91f8-15e079f0739b admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.232365 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-70cfcc6e-8040-42c6-91f8-15e079f0739b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:45.256892 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfcc6e-8040-42c6-91f8-15e079f0739b admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.265818 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.267080 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.267791 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.268296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] PATH_INFO: `/v3/projects/817282b3dff74ed8a02ed25101f21b70` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.269777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=817282b3dff74ed8a02ed25101f21b70)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.272093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.283289 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:45.337782 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1395/2800] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:45 2022] POST /identity/v3/users => generated 299 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:45.338710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c356d-9dfc-4df5-96d4-e8cc3b25de5f admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.350537 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c356d-9dfc-4df5-96d4-e8cc3b25de5f admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.358511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c212a6-b92d-4d65-9c2f-9569a7dc35e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.358511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c212a6-b92d-4d65-9c2f-9569a7dc35e6 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.358511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c212a6-b92d-4d65-9c2f-9569a7dc35e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.358511 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c212a6-b92d-4d65-9c2f-9569a7dc35e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.359204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c212a6-b92d-4d65-9c2f-9569a7dc35e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.361456 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c212a6-b92d-4d65-9c2f-9569a7dc35e6 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.377375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1396/2801] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:45.378219 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c212a6-b92d-4d65-9c2f-9569a7dc35e6 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.393320 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c212a6-b92d-4d65-9c2f-9569a7dc35e6 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.406974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e4c31-5295-43ef-a7c8-628ae200d458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.407579 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4c31-5295-43ef-a7c8-628ae200d458 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.407896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4c31-5295-43ef-a7c8-628ae200d458 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.408133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4c31-5295-43ef-a7c8-628ae200d458 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.409227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e4c31-5295-43ef-a7c8-628ae200d458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.410974 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e4c31-5295-43ef-a7c8-628ae200d458 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.449221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1397/2802] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:45.450094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e4c31-5295-43ef-a7c8-628ae200d458 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.460792 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '817282b3dff74ed8a02ed25101f21b70', 'request_id': 'req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:45.468601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '817282b3dff74ed8a02ed25101f21b70', 'request_id': 'req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:45.468922 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e4c31-5295-43ef-a7c8-628ae200d458 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.474929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '817282b3dff74ed8a02ed25101f21b70', 'request_id': 'req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:45.545486 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] The token cache is being invalidate because project 817282b3dff74ed8a02ed25101f21b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:45.545864 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:45.546788 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-9aa5ea7d-9abf-4888-9d35-2b1f13f355d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:45.553814 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1406/2803] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:45 2022] DELETE /identity/v3/projects/817282b3dff74ed8a02ed25101f21b70 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:45.557580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fbdb1-252f-4029-b83b-2a6f3c303ece None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.557868 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fbdb1-252f-4029-b83b-2a6f3c303ece None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.558151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fbdb1-252f-4029-b83b-2a6f3c303ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.562775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:45.589022 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-493fbdb1-252f-4029-b83b-2a6f3c303ece None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:45.590427 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-493fbdb1-252f-4029-b83b-2a6f3c303ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:45.602764 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e92aa-caf8-4241-ac15-3f70c8cc3d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.604168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e92aa-caf8-4241-ac15-3f70c8cc3d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.606029 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e92aa-caf8-4241-ac15-3f70c8cc3d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.606763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e92aa-caf8-4241-ac15-3f70c8cc3d0e admin admin] PATH_INFO: `/v3/projects/c450f3708f364471918691699ec92d65/users/057ec931427f432bb3b6c3a64b7a7ea8/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.667670 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e92aa-caf8-4241-ac15-3f70c8cc3d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c450f3708f364471918691699ec92d65, user_id=057ec931427f432bb3b6c3a64b7a7ea8, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.670846 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e92aa-caf8-4241-ac15-3f70c8cc3d0e admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.711927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1407/2804] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:45.713131 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b187f38-e8ff-4b3d-9c43-e59f2bf4c525 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.713896 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b187f38-e8ff-4b3d-9c43-e59f2bf4c525 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.714276 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b187f38-e8ff-4b3d-9c43-e59f2bf4c525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.751936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-4b187f38-e8ff-4b3d-9c43-e59f2bf4c525 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:45.753332 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4b187f38-e8ff-4b3d-9c43-e59f2bf4c525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:45.776594 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1408/2805] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:45.778384 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b187f38-e8ff-4b3d-9c43-e59f2bf4c525 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.798419 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b187f38-e8ff-4b3d-9c43-e59f2bf4c525 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.812112 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1398/2806] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:45 2022] PUT /identity/v3/projects/c450f3708f364471918691699ec92d65/users/057ec931427f432bb3b6c3a64b7a7ea8/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:45.814667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967ff4c-2950-4840-a317-e6c03049f432 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.821339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967ff4c-2950-4840-a317-e6c03049f432 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.821339 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967ff4c-2950-4840-a317-e6c03049f432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.875861 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203515be-3213-4eca-b53f-7c863977b5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.877646 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203515be-3213-4eca-b53f-7c863977b5ea admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.878102 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203515be-3213-4eca-b53f-7c863977b5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.879591 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203515be-3213-4eca-b53f-7c863977b5ea admin admin] PATH_INFO: `/v3/domains` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.880963 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203515be-3213-4eca-b53f-7c863977b5ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91980) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:45.881465 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203515be-3213-4eca-b53f-7c863977b5ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.887954 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203515be-3213-4eca-b53f-7c863977b5ea admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.928199 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1409/2807] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:45.929281 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203515be-3213-4eca-b53f-7c863977b5ea admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.932538 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-f967ff4c-2950-4840-a317-e6c03049f432 None None] MFA Rules not processed for user `057ec931427f432bb3b6c3a64b7a7ea8`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:45.940592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f967ff4c-2950-4840-a317-e6c03049f432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:45.942684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203515be-3213-4eca-b53f-7c863977b5ea admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.951367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dc8d6-b2d8-44fc-aa61-9107f5285653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.952149 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc8d6-b2d8-44fc-aa61-9107f5285653 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.952529 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc8d6-b2d8-44fc-aa61-9107f5285653 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.952904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc8d6-b2d8-44fc-aa61-9107f5285653 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.954521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dc8d6-b2d8-44fc-aa61-9107f5285653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.956524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dc8d6-b2d8-44fc-aa61-9107f5285653 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:46.047822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1410/2808] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:45 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:46.054574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dc8d6-b2d8-44fc-aa61-9107f5285653 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:46.061489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:46.124518 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1399/2809] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:46.152626 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967ff4c-2950-4840-a317-e6c03049f432 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:46.155991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f967ff4c-2950-4840-a317-e6c03049f432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:46.219113 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dc8d6-b2d8-44fc-aa61-9107f5285653 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:46.233878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e71bf7-c074-480a-9002-d95328079c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:46.234508 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e71bf7-c074-480a-9002-d95328079c5d admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.234805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e71bf7-c074-480a-9002-d95328079c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.235086 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e71bf7-c074-480a-9002-d95328079c5d admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.235863 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e71bf7-c074-480a-9002-d95328079c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:46.236328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967ff4c-2950-4840-a317-e6c03049f432 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:46.237946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e71bf7-c074-480a-9002-d95328079c5d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:46.323769 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1411/2810] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:46 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:46.331261 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e71bf7-c074-480a-9002-d95328079c5d admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:46.352835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac8d6d-4c5d-407a-8f36-4aa21689fd3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:46.353533 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac8d6d-4c5d-407a-8f36-4aa21689fd3e service cinder] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.353940 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac8d6d-4c5d-407a-8f36-4aa21689fd3e service cinder] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.354228 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac8d6d-4c5d-407a-8f36-4aa21689fd3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.357755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e71bf7-c074-480a-9002-d95328079c5d admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:46.368484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b7b59-532d-4f07-bf33-d771dd27f233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:46.369173 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b7b59-532d-4f07-bf33-d771dd27f233 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.369457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b7b59-532d-4f07-bf33-d771dd27f233 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.369737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b7b59-532d-4f07-bf33-d771dd27f233 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.371064 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b7b59-532d-4f07-bf33-d771dd27f233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:46.372370 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b7b59-532d-4f07-bf33-d771dd27f233 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:46.415522 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1412/2811] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:46.420805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b7b59-532d-4f07-bf33-d771dd27f233 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:46.446632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b7b59-532d-4f07-bf33-d771dd27f233 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:46.456180 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:46.463655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61183b-1e69-4240-98e1-6621257223e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:46.465550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61183b-1e69-4240-98e1-6621257223e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.465852 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61183b-1e69-4240-98e1-6621257223e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.466139 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61183b-1e69-4240-98e1-6621257223e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.469085 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61183b-1e69-4240-98e1-6621257223e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:46.471001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61183b-1e69-4240-98e1-6621257223e1 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:46.511075 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1413/2812] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:46.515699 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61183b-1e69-4240-98e1-6621257223e1 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:46.546120 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac8d6d-4c5d-407a-8f36-4aa21689fd3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:46.547639 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac8d6d-4c5d-407a-8f36-4aa21689fd3e service cinder] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:46.560750 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61183b-1e69-4240-98e1-6621257223e1 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:46.579658 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e09688-84c8-42e4-a8a5-e1012a9a7776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:46.580577 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e09688-84c8-42e4-a8a5-e1012a9a7776 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.580862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e09688-84c8-42e4-a8a5-e1012a9a7776 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.581139 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e09688-84c8-42e4-a8a5-e1012a9a7776 admin admin] PATH_INFO: `/v3/projects/05137ab7eb2c4e6080c0bbcca9de5d93/users/fef6a8f5f663414280527c5c3c525d40/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.602751 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1400/2813] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:46.675100 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e09688-84c8-42e4-a8a5-e1012a9a7776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05137ab7eb2c4e6080c0bbcca9de5d93, user_id=fef6a8f5f663414280527c5c3c525d40, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:46.683168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e09688-84c8-42e4-a8a5-e1012a9a7776 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:46.827774 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac8d6d-4c5d-407a-8f36-4aa21689fd3e service cinder] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:46.829795 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-27ac8d6d-4c5d-407a-8f36-4aa21689fd3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:46.847319 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1414/2814] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:46 2022] PUT /identity/v3/projects/05137ab7eb2c4e6080c0bbcca9de5d93/users/fef6a8f5f663414280527c5c3c525d40/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:46.859521 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788faa6c-5b8c-434c-b22c-95874eca70c6 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.860076 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788faa6c-5b8c-434c-b22c-95874eca70c6 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.860544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788faa6c-5b8c-434c-b22c-95874eca70c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.862535 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac8d6d-4c5d-407a-8f36-4aa21689fd3e service cinder] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:46.912476 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-788faa6c-5b8c-434c-b22c-95874eca70c6 None None] MFA Rules not processed for user `fef6a8f5f663414280527c5c3c525d40`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:46.915118 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-788faa6c-5b8c-434c-b22c-95874eca70c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:46.988286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:46.992097 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.992356 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.995265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] PATH_INFO: `/v3/users/8d9c651be44a47fc81d80a2448682e12` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.995265 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9c651be44a47fc81d80a2448682e12)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:46.999552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:47.048669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1415/2815] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:47.049682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788faa6c-5b8c-434c-b22c-95874eca70c6 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:47.095939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:47.135230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788faa6c-5b8c-434c-b22c-95874eca70c6 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:47.141884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977ad10-9127-44c8-9b07-3dbe6d4526b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6640e4c799514429a2fe7850369af4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:47.142607 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977ad10-9127-44c8-9b07-3dbe6d4526b7 service cinder] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:47.142867 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977ad10-9127-44c8-9b07-3dbe6d4526b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:47.143142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977ad10-9127-44c8-9b07-3dbe6d4526b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:47.156190 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977ad10-9127-44c8-9b07-3dbe6d4526b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:47.158092 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977ad10-9127-44c8-9b07-3dbe6d4526b7 service cinder] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:47.181528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1416/2816] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:47.251279 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9c651be44a47fc81d80a2448682e12'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:47.261770 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9c651be44a47fc81d80a2448682e12'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:47.324744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9c651be44a47fc81d80a2448682e12'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:47.326227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9c651be44a47fc81d80a2448682e12'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:47.326942 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9c651be44a47fc81d80a2448682e12'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:47.382181 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1401/2817] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:46 2022] DELETE /identity/v3/users/8d9c651be44a47fc81d80a2448682e12 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:47.387554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03538447-8a92-4b1a-b258-44ecebbbdf97 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:47.387554 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03538447-8a92-4b1a-b258-44ecebbbdf97 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:47.387817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03538447-8a92-4b1a-b258-44ecebbbdf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:47.412789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-03538447-8a92-4b1a-b258-44ecebbbdf97 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:47.414163 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-03538447-8a92-4b1a-b258-44ecebbbdf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:47.566626 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1417/2818] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:47.571309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:47.573465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:47.750855 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc46d2-8ea3-4046-8965-3620afe65a05 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:47.757118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53938fa6-29a3-46cf-9bed-9ef165033175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:47.758194 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53938fa6-29a3-46cf-9bed-9ef165033175 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:47.758657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53938fa6-29a3-46cf-9bed-9ef165033175 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:47.758944 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53938fa6-29a3-46cf-9bed-9ef165033175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:47.778100 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:47.782739 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53938fa6-29a3-46cf-9bed-9ef165033175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:47.784571 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53938fa6-29a3-46cf-9bed-9ef165033175 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:47.803572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1402/2819] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:48.405888 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03538447-8a92-4b1a-b258-44ecebbbdf97 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:48.407868 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-03538447-8a92-4b1a-b258-44ecebbbdf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:48.426093 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03538447-8a92-4b1a-b258-44ecebbbdf97 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:48.432124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:48.433155 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:48.433537 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:48.433929 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] PATH_INFO: `/v3/projects/91a61dfae6db4d46878906d2038f2a6d` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:48.435343 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a61dfae6db4d46878906d2038f2a6d)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:48.437822 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:49.061363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a61dfae6db4d46878906d2038f2a6d', 'request_id': 'req-10e48516-ecbf-4622-9cca-e2bf0e85473d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.067938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a61dfae6db4d46878906d2038f2a6d', 'request_id': 'req-10e48516-ecbf-4622-9cca-e2bf0e85473d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.074499 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a61dfae6db4d46878906d2038f2a6d', 'request_id': 'req-10e48516-ecbf-4622-9cca-e2bf0e85473d'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.162258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] The token cache is being invalidate because project 91a61dfae6db4d46878906d2038f2a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:49.162698 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.163415 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.169629 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1418/2820] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:48 2022] DELETE /identity/v3/projects/91a61dfae6db4d46878906d2038f2a6d => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:49.174011 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53938fa6-29a3-46cf-9bed-9ef165033175 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:49.185302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53938fa6-29a3-46cf-9bed-9ef165033175 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:49.282127 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:49.283360 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.283906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.284343 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] PATH_INFO: `/v3/users/d01db669717841059bd13216750e5597` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.285729 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d01db669717841059bd13216750e5597)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:49.288090 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:49.546704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd01db669717841059bd13216750e5597'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.557074 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd01db669717841059bd13216750e5597'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.643812 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd01db669717841059bd13216750e5597'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.644467 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd01db669717841059bd13216750e5597'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.645118 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-e7fee23d-0c2c-4306-9358-bf5724256375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd01db669717841059bd13216750e5597'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.708545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1403/2821] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:49 2022] DELETE /identity/v3/users/d01db669717841059bd13216750e5597 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:50.183454 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.185586 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:50.201684 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e48516-ecbf-4622-9cca-e2bf0e85473d admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.273194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.273447 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.273754 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.274580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] PATH_INFO: `/v3/projects/c9a98126fca5457999d2fbaa0fb66cbb` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.275885 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a98126fca5457999d2fbaa0fb66cbb)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.278123 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.280809 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:50.393478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a98126fca5457999d2fbaa0fb66cbb', 'request_id': 'req-2da9e244-8153-4f86-b03a-d597706e3a9c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:50.404224 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a98126fca5457999d2fbaa0fb66cbb', 'request_id': 'req-2da9e244-8153-4f86-b03a-d597706e3a9c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:50.410851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a98126fca5457999d2fbaa0fb66cbb', 'request_id': 'req-2da9e244-8153-4f86-b03a-d597706e3a9c'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:50.478081 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] The token cache is being invalidate because project c9a98126fca5457999d2fbaa0fb66cbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:50.478441 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:50.486293 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-2da9e244-8153-4f86-b03a-d597706e3a9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:50.488496 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1419/2822] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:50 2022] DELETE /identity/v3/projects/c9a98126fca5457999d2fbaa0fb66cbb => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:50.493970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d84457-c431-42b2-aa3f-a1e800a3ec82 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.494506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d84457-c431-42b2-aa3f-a1e800a3ec82 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.494612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d84457-c431-42b2-aa3f-a1e800a3ec82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.514890 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-e0d84457-c431-42b2-aa3f-a1e800a3ec82 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:50.516187 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e0d84457-c431-42b2-aa3f-a1e800a3ec82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:50.604020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1404/2823] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:50.612150 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c61c-0149-4c85-98ee-563d1f39dfde None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.612580 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c61c-0149-4c85-98ee-563d1f39dfde None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.612949 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c61c-0149-4c85-98ee-563d1f39dfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.641176 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-c0a3c61c-0149-4c85-98ee-563d1f39dfde None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:50.642391 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c0a3c61c-0149-4c85-98ee-563d1f39dfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:50.654220 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1420/2824] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:50.655217 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d84457-c431-42b2-aa3f-a1e800a3ec82 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.668076 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d84457-c431-42b2-aa3f-a1e800a3ec82 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.676038 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36640392-e26e-420e-9cf3-da6db91c0198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.676722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36640392-e26e-420e-9cf3-da6db91c0198 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.677009 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36640392-e26e-420e-9cf3-da6db91c0198 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.677286 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36640392-e26e-420e-9cf3-da6db91c0198 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.681528 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36640392-e26e-420e-9cf3-da6db91c0198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:50.681925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36640392-e26e-420e-9cf3-da6db91c0198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.683615 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36640392-e26e-420e-9cf3-da6db91c0198 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.697993 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1405/2825] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:06:50.702830 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3c61c-0149-4c85-98ee-563d1f39dfde None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.714066 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3c61c-0149-4c85-98ee-563d1f39dfde None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.723840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31ee7f-a902-4407-86bf-7ce5feb57f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.724530 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31ee7f-a902-4407-86bf-7ce5feb57f4f admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.724807 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31ee7f-a902-4407-86bf-7ce5feb57f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.725878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31ee7f-a902-4407-86bf-7ce5feb57f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.725878 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31ee7f-a902-4407-86bf-7ce5feb57f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.728587 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31ee7f-a902-4407-86bf-7ce5feb57f4f admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.819048 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1421/2826] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:50 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:50.823261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36640392-e26e-420e-9cf3-da6db91c0198 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.828870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:50.835452 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36640392-e26e-420e-9cf3-da6db91c0198 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.915372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac753e4a-0679-441c-9d68-d3cf3e071e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.916283 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac753e4a-0679-441c-9d68-d3cf3e071e9b admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.916857 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac753e4a-0679-441c-9d68-d3cf3e071e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.917549 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac753e4a-0679-441c-9d68-d3cf3e071e9b admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.918634 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac753e4a-0679-441c-9d68-d3cf3e071e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.921484 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac753e4a-0679-441c-9d68-d3cf3e071e9b admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.006611 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1406/2827] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:50 2022] POST /identity/v3/users => generated 284 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:51.008844 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31ee7f-a902-4407-86bf-7ce5feb57f4f admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.019546 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31ee7f-a902-4407-86bf-7ce5feb57f4f admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.026463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ec5a3-5fae-412a-b804-affed19b5039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.026463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ec5a3-5fae-412a-b804-affed19b5039 admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.026463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ec5a3-5fae-412a-b804-affed19b5039 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.026463 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ec5a3-5fae-412a-b804-affed19b5039 admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.026894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ec5a3-5fae-412a-b804-affed19b5039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.029615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ec5a3-5fae-412a-b804-affed19b5039 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.040588 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1422/2828] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:51.045209 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac753e4a-0679-441c-9d68-d3cf3e071e9b admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.056921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac753e4a-0679-441c-9d68-d3cf3e071e9b admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.061775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6658c-8222-4d0f-ae55-4b284c51ceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.063054 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6658c-8222-4d0f-ae55-4b284c51ceaa admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.063346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6658c-8222-4d0f-ae55-4b284c51ceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.063623 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6658c-8222-4d0f-ae55-4b284c51ceaa admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.064413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6658c-8222-4d0f-ae55-4b284c51ceaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.066018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6658c-8222-4d0f-ae55-4b284c51ceaa admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.083984 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1407/2829] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:51.088192 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ec5a3-5fae-412a-b804-affed19b5039 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.099922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ec5a3-5fae-412a-b804-affed19b5039 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.106001 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebf010-54f4-4a50-a77a-cc70294bdb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.107314 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebf010-54f4-4a50-a77a-cc70294bdb5a admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.107314 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebf010-54f4-4a50-a77a-cc70294bdb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.108533 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebf010-54f4-4a50-a77a-cc70294bdb5a admin admin] PATH_INFO: `/v3/projects/39de1064c8fd405aa4867d64bfbbb55d/users/257cdd2dcc71472c8c73707526125e64/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.125740 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebf010-54f4-4a50-a77a-cc70294bdb5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=39de1064c8fd405aa4867d64bfbbb55d, user_id=257cdd2dcc71472c8c73707526125e64, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.129515 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebf010-54f4-4a50-a77a-cc70294bdb5a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.207152 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1423/2830] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:51 2022] PUT /identity/v3/projects/39de1064c8fd405aa4867d64bfbbb55d/users/257cdd2dcc71472c8c73707526125e64/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:51.217704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e3ff4-5cb6-4efd-9384-6e5760e0fdcd None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.218212 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e3ff4-5cb6-4efd-9384-6e5760e0fdcd None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.218651 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e3ff4-5cb6-4efd-9384-6e5760e0fdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.254501 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-1e7e3ff4-5cb6-4efd-9384-6e5760e0fdcd None None] MFA Rules not processed for user `257cdd2dcc71472c8c73707526125e64`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:51.255749 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-1e7e3ff4-5cb6-4efd-9384-6e5760e0fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:51.332238 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1408/2831] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:51.346610 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebf010-54f4-4a50-a77a-cc70294bdb5a admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.348594 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-1eebf010-54f4-4a50-a77a-cc70294bdb5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:51.419935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebf010-54f4-4a50-a77a-cc70294bdb5a admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.424394 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216a1b0-d0dc-45e5-aa6d-0c169b3a4d47 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e12361da5f849ee97607ca68ee6ef10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.425158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216a1b0-d0dc-45e5-aa6d-0c169b3a4d47 service glance] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.425507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216a1b0-d0dc-45e5-aa6d-0c169b3a4d47 service glance] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.425862 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216a1b0-d0dc-45e5-aa6d-0c169b3a4d47 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.435177 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216a1b0-d0dc-45e5-aa6d-0c169b3a4d47 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.436977 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216a1b0-d0dc-45e5-aa6d-0c169b3a4d47 service glance] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.450544 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1424/2832] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:56.178042 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283c188-7464-417a-9230-b741343952eb None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:56.178681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283c188-7464-417a-9230-b741343952eb None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:56.178681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283c188-7464-417a-9230-b741343952eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:56.196728 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-3283c188-7464-417a-9230-b741343952eb None None] MFA Rules not processed for user `a01a4b9868714295a1000e0a5a81143b`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:56.197951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3283c188-7464-417a-9230-b741343952eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:56.262567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1409/2833] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:57.095284 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216a1b0-d0dc-45e5-aa6d-0c169b3a4d47 service glance] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:57.097582 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8216a1b0-d0dc-45e5-aa6d-0c169b3a4d47 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:57.132939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:57.162969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216a1b0-d0dc-45e5-aa6d-0c169b3a4d47 service glance] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:57.168194 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:57.169390 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:57.169604 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:57.169883 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] PATH_INFO: `/v3/users/057ec931427f432bb3b6c3a64b7a7ea8` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:57.171151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=057ec931427f432bb3b6c3a64b7a7ea8)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:57.173309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:57.386288 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '057ec931427f432bb3b6c3a64b7a7ea8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:57.393668 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '057ec931427f432bb3b6c3a64b7a7ea8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:57.394349 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '057ec931427f432bb3b6c3a64b7a7ea8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:57.466566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '057ec931427f432bb3b6c3a64b7a7ea8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:57.466981 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '057ec931427f432bb3b6c3a64b7a7ea8'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:57.495445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1425/2834] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:57 2022] DELETE /identity/v3/users/057ec931427f432bb3b6c3a64b7a7ea8 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:57.500405 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01bd10-00f4-4631-93b3-1d598fa96604 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:57.501145 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01bd10-00f4-4631-93b3-1d598fa96604 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:57.501250 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01bd10-00f4-4631-93b3-1d598fa96604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:57.519905 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-0f01bd10-00f4-4631-93b3-1d598fa96604 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:57.521113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0f01bd10-00f4-4631-93b3-1d598fa96604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:57.531331 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:57.583962 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1410/2835] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:57.594290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:57.595764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:57.678808 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8487446-2761-4a87-a191-22bcce5e1fb3 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:57.683924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31652d2-0516-4034-8397-89dc7ee73c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:57.686620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31652d2-0516-4034-8397-89dc7ee73c2b service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:57.686620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31652d2-0516-4034-8397-89dc7ee73c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:57.686620 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31652d2-0516-4034-8397-89dc7ee73c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:57.694449 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31652d2-0516-4034-8397-89dc7ee73c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:57.695941 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31652d2-0516-4034-8397-89dc7ee73c2b service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:57.706502 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1426/2836] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:58.187583 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01bd10-00f4-4631-93b3-1d598fa96604 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:58.189381 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-0f01bd10-00f4-4631-93b3-1d598fa96604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:58.206122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01bd10-00f4-4631-93b3-1d598fa96604 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:58.212130 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:58.213723 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:58.213894 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:58.214305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] PATH_INFO: `/v3/projects/c450f3708f364471918691699ec92d65` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:58.219713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c450f3708f364471918691699ec92d65)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:58.219713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:58.342454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c450f3708f364471918691699ec92d65', 'request_id': 'req-86346bae-3230-42f6-ba20-545babc2becf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:58.348684 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c450f3708f364471918691699ec92d65', 'request_id': 'req-86346bae-3230-42f6-ba20-545babc2becf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:58.354431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c450f3708f364471918691699ec92d65', 'request_id': 'req-86346bae-3230-42f6-ba20-545babc2becf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:58.442802 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] The token cache is being invalidate because project c450f3708f364471918691699ec92d65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:58.448726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:58.448726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-86346bae-3230-42f6-ba20-545babc2becf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:58.448726 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1411/2837] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:58 2022] DELETE /identity/v3/projects/c450f3708f364471918691699ec92d65 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:59.676928 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31652d2-0516-4034-8397-89dc7ee73c2b service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:59.678405 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d31652d2-0516-4034-8397-89dc7ee73c2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:59.692829 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31652d2-0516-4034-8397-89dc7ee73c2b service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:59.729426 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:59.762578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:59.767214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:59.767214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:59.767214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] PATH_INFO: `/v3/users/fef6a8f5f663414280527c5c3c525d40` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:59.767214 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef6a8f5f663414280527c5c3c525d40)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:59.767572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:59.878676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c6d75-2ede-4750-abce-817f99be5f0d None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:59.879144 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c6d75-2ede-4750-abce-817f99be5f0d None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:59.879261 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c6d75-2ede-4750-abce-817f99be5f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:59.900827 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-d15c6d75-2ede-4750-abce-817f99be5f0d None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:59.902980 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d15c6d75-2ede-4750-abce-817f99be5f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:59.974978 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1412/2838] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Jul 25 06:06:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:59.987346 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c6d75-2ede-4750-abce-817f99be5f0d None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:59.988740 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-d15c6d75-2ede-4750-abce-817f99be5f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:00.038036 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:00.052304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef6a8f5f663414280527c5c3c525d40'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:00.065202 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef6a8f5f663414280527c5c3c525d40'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:00.068585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef6a8f5f663414280527c5c3c525d40'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:00.077609 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c6d75-2ede-4750-abce-817f99be5f0d None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:00.087169 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df354c4-5e38-47b6-be2e-22349b2d83d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:00.088304 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df354c4-5e38-47b6-be2e-22349b2d83d2 service swift] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:00.088799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df354c4-5e38-47b6-be2e-22349b2d83d2 service swift] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:00.089241 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df354c4-5e38-47b6-be2e-22349b2d83d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:00.102760 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df354c4-5e38-47b6-be2e-22349b2d83d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:00.105780 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df354c4-5e38-47b6-be2e-22349b2d83d2 service swift] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:00.115713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1413/2839] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:06:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 06:07:00.158853 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef6a8f5f663414280527c5c3c525d40'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:00.161128 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef6a8f5f663414280527c5c3c525d40'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:00.182726 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1427/2840] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:59 2022] DELETE /identity/v3/users/fef6a8f5f663414280527c5c3c525d40 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:00.185671 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add7c4-62bf-4956-aced-912669b5c550 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:00.185982 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add7c4-62bf-4956-aced-912669b5c550 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:00.186260 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add7c4-62bf-4956-aced-912669b5c550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:00.205219 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-11add7c4-62bf-4956-aced-912669b5c550 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:00.206553 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-11add7c4-62bf-4956-aced-912669b5c550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:00.306530 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1414/2841] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:07:00.324191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:00.326183 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:00.475314 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4e5a1-c186-4639-a244-8cbb3cd47317 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:00.485290 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2dbe1-128a-4446-a972-4b1923abec76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:00.486812 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2dbe1-128a-4446-a972-4b1923abec76 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:00.489164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2dbe1-128a-4446-a972-4b1923abec76 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:00.489618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2dbe1-128a-4446-a972-4b1923abec76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:00.515819 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2dbe1-128a-4446-a972-4b1923abec76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:00.520665 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2dbe1-128a-4446-a972-4b1923abec76 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:00.538993 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1428/2842] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:01.349556 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11add7c4-62bf-4956-aced-912669b5c550 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:01.351161 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-11add7c4-62bf-4956-aced-912669b5c550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:01.380449 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11add7c4-62bf-4956-aced-912669b5c550 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:01.395344 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:01.396313 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:01.396541 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:01.396832 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] PATH_INFO: `/v3/projects/05137ab7eb2c4e6080c0bbcca9de5d93` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:01.398142 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05137ab7eb2c4e6080c0bbcca9de5d93)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:01.401612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:01.600133 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05137ab7eb2c4e6080c0bbcca9de5d93', 'request_id': 'req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:01.622502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05137ab7eb2c4e6080c0bbcca9de5d93', 'request_id': 'req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:01.640048 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05137ab7eb2c4e6080c0bbcca9de5d93', 'request_id': 'req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:01.759464 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] The token cache is being invalidate because project 05137ab7eb2c4e6080c0bbcca9de5d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:01.761570 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:01.770701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:01.770701 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1415/2843] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:01 2022] DELETE /identity/v3/projects/05137ab7eb2c4e6080c0bbcca9de5d93 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:03.110799 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2dbe1-128a-4446-a972-4b1923abec76 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:03.125601 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2dbe1-128a-4446-a972-4b1923abec76 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:03.216939 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bf879-e77b-41a3-b7c8-dfb96982d967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:03.218079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bf879-e77b-41a3-b7c8-dfb96982d967 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:03.218079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bf879-e77b-41a3-b7c8-dfb96982d967 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:03.218367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bf879-e77b-41a3-b7c8-dfb96982d967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:03.220731 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-323bf879-e77b-41a3-b7c8-dfb96982d967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:03.230895 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:03.293260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bf879-e77b-41a3-b7c8-dfb96982d967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:03.295053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bf879-e77b-41a3-b7c8-dfb96982d967 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:03.306363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1429/2844] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:11.724744 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.726371 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:11.789806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:11.823385 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f27c0-0621-469e-ae79-b57d3ec9c5b7 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.828817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630989d-3737-4227-b83b-f37be43f31f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.829463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630989d-3737-4227-b83b-f37be43f31f9 service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.829746 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630989d-3737-4227-b83b-f37be43f31f9 service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.830032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630989d-3737-4227-b83b-f37be43f31f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.831837 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f630989d-3737-4227-b83b-f37be43f31f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:11.844420 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630989d-3737-4227-b83b-f37be43f31f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.846008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630989d-3737-4227-b83b-f37be43f31f9 service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.856564 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1416/2845] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:13.334848 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=23 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:19.190259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bf879-e77b-41a3-b7c8-dfb96982d967 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:19.191869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-323bf879-e77b-41a3-b7c8-dfb96982d967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:19.203424 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bf879-e77b-41a3-b7c8-dfb96982d967 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:19.207117 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590b56b-6754-4ea7-8ca7-df58a8f14e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:19.207760 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590b56b-6754-4ea7-8ca7-df58a8f14e99 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:19.208038 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590b56b-6754-4ea7-8ca7-df58a8f14e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:19.208311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590b56b-6754-4ea7-8ca7-df58a8f14e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:19.211575 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f590b56b-6754-4ea7-8ca7-df58a8f14e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:19.274050 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590b56b-6754-4ea7-8ca7-df58a8f14e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:19.275524 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590b56b-6754-4ea7-8ca7-df58a8f14e99 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:19.285543 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1430/2846] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:07:22.065665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=17 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:26.542249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630989d-3737-4227-b83b-f37be43f31f9 service nova] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:26.549761 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-f630989d-3737-4227-b83b-f37be43f31f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:26.562017 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630989d-3737-4227-b83b-f37be43f31f9 service nova] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:26.566799 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:26.567879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:26.568164 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:26.568499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] PATH_INFO: `/v3/users/d03b404df6d54b638804804bbc08e1cf` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:26.570018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03b404df6d54b638804804bbc08e1cf)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:26.572544 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:26.925800 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03b404df6d54b638804804bbc08e1cf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:26.932499 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03b404df6d54b638804804bbc08e1cf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.037155 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03b404df6d54b638804804bbc08e1cf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.037918 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03b404df6d54b638804804bbc08e1cf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.038661 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03b404df6d54b638804804bbc08e1cf'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.060971 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1417/2847] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:26 2022] DELETE /identity/v3/users/d03b404df6d54b638804804bbc08e1cf => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:27.296277 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590b56b-6754-4ea7-8ca7-df58a8f14e99 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:27.306495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590b56b-6754-4ea7-8ca7-df58a8f14e99 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:27.361874 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:27.362771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:27.363052 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:27.363325 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] PATH_INFO: `/v3/projects/52a1423a26e041959827c5d8611cffdf` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:27.364495 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52a1423a26e041959827c5d8611cffdf)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:27.366368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:27.483489 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52a1423a26e041959827c5d8611cffdf', 'request_id': 'req-c334ee58-8b81-477f-8fb9-a4d4faa32880'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.489648 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52a1423a26e041959827c5d8611cffdf', 'request_id': 'req-c334ee58-8b81-477f-8fb9-a4d4faa32880'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.495084 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52a1423a26e041959827c5d8611cffdf', 'request_id': 'req-c334ee58-8b81-477f-8fb9-a4d4faa32880'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.556270 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] The token cache is being invalidate because project 52a1423a26e041959827c5d8611cffdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:27.556559 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.557181 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.558771 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1431/2848] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:27 2022] DELETE /identity/v3/projects/52a1423a26e041959827c5d8611cffdf => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:27.562896 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:27.564297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:27.577934 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44388187-e6c8-42ff-bd9c-271d4b11e023 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:27.637204 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:27.638094 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:27.638383 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:27.638698 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] PATH_INFO: `/v3/users/3b52c97f202c4bff83ec4875658b6e5b` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:27.639950 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b52c97f202c4bff83ec4875658b6e5b)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:27.641925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:27.834124 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b52c97f202c4bff83ec4875658b6e5b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.841417 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b52c97f202c4bff83ec4875658b6e5b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.882925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b52c97f202c4bff83ec4875658b6e5b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.883554 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b52c97f202c4bff83ec4875658b6e5b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.884168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b52c97f202c4bff83ec4875658b6e5b'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:27.892375 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:27.902355 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1418/2849] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:27 2022] DELETE /identity/v3/users/3b52c97f202c4bff83ec4875658b6e5b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:28.141987 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:28.143734 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:28.157905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334ee58-8b81-477f-8fb9-a4d4faa32880 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:28.180182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:28.218241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:28.219141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:28.219422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:28.219694 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] PATH_INFO: `/v3/projects/f339b1033b2948bea1d9fe6eb1f387b9` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:28.220857 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f339b1033b2948bea1d9fe6eb1f387b9)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:28.222764 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:28.328669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f339b1033b2948bea1d9fe6eb1f387b9', 'request_id': 'req-4f404dc2-cd5a-40f0-92be-d5e1d1238593'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:28.336306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f339b1033b2948bea1d9fe6eb1f387b9', 'request_id': 'req-4f404dc2-cd5a-40f0-92be-d5e1d1238593'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:28.342455 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f339b1033b2948bea1d9fe6eb1f387b9', 'request_id': 'req-4f404dc2-cd5a-40f0-92be-d5e1d1238593'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:28.431158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] The token cache is being invalidate because project f339b1033b2948bea1d9fe6eb1f387b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:28.431452 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:28.432114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-4f404dc2-cd5a-40f0-92be-d5e1d1238593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:28.433664 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1432/2850] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:28 2022] DELETE /identity/v3/projects/f339b1033b2948bea1d9fe6eb1f387b9 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:29.403491 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:29.404909 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:29.417567 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27a027-e055-46e4-b2ce-aefac2385780 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:29.472526 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:29.473394 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:29.473681 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:29.473951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] PATH_INFO: `/v3/users/257cdd2dcc71472c8c73707526125e64` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:29.475134 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=257cdd2dcc71472c8c73707526125e64)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:29.477049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:29.656466 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '257cdd2dcc71472c8c73707526125e64'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:29.663599 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '257cdd2dcc71472c8c73707526125e64'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:29.745755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '257cdd2dcc71472c8c73707526125e64'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:29.746384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '257cdd2dcc71472c8c73707526125e64'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:29.747618 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '257cdd2dcc71472c8c73707526125e64'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:29.765742 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1419/2851] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:29 2022] DELETE /identity/v3/users/257cdd2dcc71472c8c73707526125e64 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:29.770267 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255a757-3a8e-47ad-97b1-716c0848e345 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:29.770794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255a757-3a8e-47ad-97b1-716c0848e345 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:29.770901 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255a757-3a8e-47ad-97b1-716c0848e345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:29.788260 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-f255a757-3a8e-47ad-97b1-716c0848e345 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:29.789448 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f255a757-3a8e-47ad-97b1-716c0848e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:29.857035 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1433/2852] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:29.867299 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:29.868689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:29.927281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c860255-5ab4-4c31-bd3f-6d888b5185c4 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:29.931002 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e329a-e261-4cd9-acfe-282ddd4823c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:29.931647 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e329a-e261-4cd9-acfe-282ddd4823c7 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:29.931925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e329a-e261-4cd9-acfe-282ddd4823c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:29.932197 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e329a-e261-4cd9-acfe-282ddd4823c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:29.941565 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e329a-e261-4cd9-acfe-282ddd4823c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:29.943030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e329a-e261-4cd9-acfe-282ddd4823c7 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:29.952240 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1420/2853] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:30.342196 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255a757-3a8e-47ad-97b1-716c0848e345 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:30.343694 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f255a757-3a8e-47ad-97b1-716c0848e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:30.355481 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255a757-3a8e-47ad-97b1-716c0848e345 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:30.360763 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:30.361728 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:30.361938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:30.362296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] PATH_INFO: `/v3/projects/39de1064c8fd405aa4867d64bfbbb55d` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:30.363563 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] RBAC: Authorizing `identity:delete_project(project_id=39de1064c8fd405aa4867d64bfbbb55d)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:30.365494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:30.489168 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39de1064c8fd405aa4867d64bfbbb55d', 'request_id': 'req-a86ae13d-eee4-4992-b701-35f5d2fdf09a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:30.490222 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:30.495725 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39de1064c8fd405aa4867d64bfbbb55d', 'request_id': 'req-a86ae13d-eee4-4992-b701-35f5d2fdf09a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:30.501759 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39de1064c8fd405aa4867d64bfbbb55d', 'request_id': 'req-a86ae13d-eee4-4992-b701-35f5d2fdf09a'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:30.597206 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] The token cache is being invalidate because project 39de1064c8fd405aa4867d64bfbbb55d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91980) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:30.597503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:30.598136 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-a86ae13d-eee4-4992-b701-35f5d2fdf09a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:30.599709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1434/2854] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:30 2022] DELETE /identity/v3/projects/39de1064c8fd405aa4867d64bfbbb55d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:30.606845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2f4af-0bff-42dc-935a-b53da8bd51e0 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:30.606845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2f4af-0bff-42dc-935a-b53da8bd51e0 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:30.606845 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2f4af-0bff-42dc-935a-b53da8bd51e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:30.625190 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-71e2f4af-0bff-42dc-935a-b53da8bd51e0 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:30.626400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-71e2f4af-0bff-42dc-935a-b53da8bd51e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:30.634991 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:30.687506 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1421/2855] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:30.696367 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e5db4-a9ee-47af-bfae-927fad36c9ec None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:30.696655 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e5db4-a9ee-47af-bfae-927fad36c9ec None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:30.697032 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e5db4-a9ee-47af-bfae-927fad36c9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:30.715212 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-835e5db4-a9ee-47af-bfae-927fad36c9ec None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:30.716428 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-835e5db4-a9ee-47af-bfae-927fad36c9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:30.724457 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1435/2856] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:30.728641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2f4af-0bff-42dc-935a-b53da8bd51e0 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:30.741105 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2f4af-0bff-42dc-935a-b53da8bd51e0 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:30.745336 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9304a91-caf8-4418-b4f1-564daae52062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:30.746008 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304a91-caf8-4418-b4f1-564daae52062 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:30.746297 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304a91-caf8-4418-b4f1-564daae52062 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:30.747226 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304a91-caf8-4418-b4f1-564daae52062 admin admin] PATH_INFO: `/v3/domains` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:30.747967 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9304a91-caf8-4418-b4f1-564daae52062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91981) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:07:30.748275 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9304a91-caf8-4418-b4f1-564daae52062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:30.749911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9304a91-caf8-4418-b4f1-564daae52062 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:30.761690 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1422/2857] 10.222.0.17 () {58 vars in 1152 bytes} [Mon Jul 25 06:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:07:30.766060 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e5db4-a9ee-47af-bfae-927fad36c9ec None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:30.775507 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e5db4-a9ee-47af-bfae-927fad36c9ec None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:30.780275 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c1ddf-58e3-46f5-857c-cb5e110d92e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:30.781462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c1ddf-58e3-46f5-857c-cb5e110d92e7 admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:30.781462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c1ddf-58e3-46f5-857c-cb5e110d92e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:30.781462 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c1ddf-58e3-46f5-857c-cb5e110d92e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:30.782142 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c1ddf-58e3-46f5-857c-cb5e110d92e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:30.784068 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c1ddf-58e3-46f5-857c-cb5e110d92e7 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:30.855247 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1436/2858] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:30 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:30.859920 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9304a91-caf8-4418-b4f1-564daae52062 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:30.869965 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9304a91-caf8-4418-b4f1-564daae52062 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:30.938542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413deab6-0810-41ac-a9a0-dca4bbf76b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:30.938542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413deab6-0810-41ac-a9a0-dca4bbf76b09 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:30.938542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413deab6-0810-41ac-a9a0-dca4bbf76b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:30.938542 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413deab6-0810-41ac-a9a0-dca4bbf76b09 admin admin] PATH_INFO: `/v3/users` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:30.938970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413deab6-0810-41ac-a9a0-dca4bbf76b09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:30.938970 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413deab6-0810-41ac-a9a0-dca4bbf76b09 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:31.043281 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1423/2859] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:30 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:31.048140 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c1ddf-58e3-46f5-857c-cb5e110d92e7 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:31.058595 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c1ddf-58e3-46f5-857c-cb5e110d92e7 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:31.063182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a1b2c-6de5-419c-9680-1ee6aacbb13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:31.063916 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a1b2c-6de5-419c-9680-1ee6aacbb13a admin admin] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:31.064204 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a1b2c-6de5-419c-9680-1ee6aacbb13a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:31.064478 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a1b2c-6de5-419c-9680-1ee6aacbb13a admin admin] PATH_INFO: `/v3/roles` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:31.065197 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a1b2c-6de5-419c-9680-1ee6aacbb13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:31.066805 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a1b2c-6de5-419c-9680-1ee6aacbb13a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:31.077422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1437/2860] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:31.081390 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413deab6-0810-41ac-a9a0-dca4bbf76b09 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:31.092082 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413deab6-0810-41ac-a9a0-dca4bbf76b09 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:31.096665 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915e23a-26aa-4769-8715-f3037827c82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:31.097316 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e23a-26aa-4769-8715-f3037827c82c admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:31.097600 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e23a-26aa-4769-8715-f3037827c82c admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:31.097873 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e23a-26aa-4769-8715-f3037827c82c admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:31.098707 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915e23a-26aa-4769-8715-f3037827c82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:31.100309 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915e23a-26aa-4769-8715-f3037827c82c admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:31.111572 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1424/2861] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:31.115674 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a1b2c-6de5-419c-9680-1ee6aacbb13a admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:31.125845 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a1b2c-6de5-419c-9680-1ee6aacbb13a admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:31.130618 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846a5d8-bdb6-4d13-8e03-914edc7157e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:31.131503 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846a5d8-bdb6-4d13-8e03-914edc7157e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:31.131790 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846a5d8-bdb6-4d13-8e03-914edc7157e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:31.132070 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846a5d8-bdb6-4d13-8e03-914edc7157e3 admin admin] PATH_INFO: `/v3/projects/5eef98733468492383e255f9045cd025/users/8650e8e864f045c59cf38e7a50949b94/roles/5dce8d92cffa40f79dfa1c03724c47e2` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:31.146398 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846a5d8-bdb6-4d13-8e03-914edc7157e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eef98733468492383e255f9045cd025, user_id=8650e8e864f045c59cf38e7a50949b94, role_id=5dce8d92cffa40f79dfa1c03724c47e2)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:31.149195 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846a5d8-bdb6-4d13-8e03-914edc7157e3 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:31.231264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1438/2862] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:31 2022] PUT /identity/v3/projects/5eef98733468492383e255f9045cd025/users/8650e8e864f045c59cf38e7a50949b94/roles/5dce8d92cffa40f79dfa1c03724c47e2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:31.236683 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4af71-de4a-4503-b893-857592d50517 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:31.236972 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4af71-de4a-4503-b893-857592d50517 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:31.237248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4af71-de4a-4503-b893-857592d50517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:31.255084 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-53a4af71-de4a-4503-b893-857592d50517 None None] MFA Rules not processed for user `16be6f6629504bf5b97a9530841ecf24`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:31.256400 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-53a4af71-de4a-4503-b893-857592d50517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:31.317376 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1425/2863] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:31.327656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846a5d8-bdb6-4d13-8e03-914edc7157e3 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:31.332704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4846a5d8-bdb6-4d13-8e03-914edc7157e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:31.389879 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846a5d8-bdb6-4d13-8e03-914edc7157e3 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:31.393813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935cebe6-fdf9-4166-9087-27849eb41498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:31.394500 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cebe6-fdf9-4166-9087-27849eb41498 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:31.396651 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cebe6-fdf9-4166-9087-27849eb41498 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:31.396938 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cebe6-fdf9-4166-9087-27849eb41498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:31.412953 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cebe6-fdf9-4166-9087-27849eb41498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:31.414609 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cebe6-fdf9-4166-9087-27849eb41498 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:31.424787 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1439/2864] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:32.279817 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4af71-de4a-4503-b893-857592d50517 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:32.289463 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4af71-de4a-4503-b893-857592d50517 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:32.293117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9389f21-9204-4137-a447-19ffbfed0c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:32.294256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9389f21-9204-4137-a447-19ffbfed0c3a service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:32.294594 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9389f21-9204-4137-a447-19ffbfed0c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:32.295131 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9389f21-9204-4137-a447-19ffbfed0c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:32.296949 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-e9389f21-9204-4137-a447-19ffbfed0c3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:32.316018 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:32.362604 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9389f21-9204-4137-a447-19ffbfed0c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:32.364096 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9389f21-9204-4137-a447-19ffbfed0c3a service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:32.374657 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1426/2865] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:37.681445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b606ea9-f6b8-4655-8ce2-1491cca25156 None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:37.682304 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b606ea9-f6b8-4655-8ce2-1491cca25156 None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:37.682632 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b606ea9-f6b8-4655-8ce2-1491cca25156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:37.716132 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-8b606ea9-f6b8-4655-8ce2-1491cca25156 None None] MFA Rules not processed for user `8650e8e864f045c59cf38e7a50949b94`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:37.717409 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8b606ea9-f6b8-4655-8ce2-1491cca25156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:37.786484 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1440/2866] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:07:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:37.791364 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9389f21-9204-4137-a447-19ffbfed0c3a service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:37.803073 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9389f21-9204-4137-a447-19ffbfed0c3a service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:37.807809 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbbad7-bb3a-49ea-9139-08a0dca6f5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:37.808722 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbbad7-bb3a-49ea-9139-08a0dca6f5b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:37.809249 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbbad7-bb3a-49ea-9139-08a0dca6f5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:37.809757 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbbad7-bb3a-49ea-9139-08a0dca6f5b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:37.810938 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbbad7-bb3a-49ea-9139-08a0dca6f5b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:37.816330 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbbad7-bb3a-49ea-9139-08a0dca6f5b8 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:37.903983 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1427/2867] 10.222.0.17 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:37 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:07:37.908063 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b606ea9-f6b8-4655-8ce2-1491cca25156 None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:37.921656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b606ea9-f6b8-4655-8ce2-1491cca25156 None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:37.926574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:38.000574 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0be6a3-81df-41c6-8727-aa90713f289a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:38.001255 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0be6a3-81df-41c6-8727-aa90713f289a admin admin] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:38.001539 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0be6a3-81df-41c6-8727-aa90713f289a admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:38.001813 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0be6a3-81df-41c6-8727-aa90713f289a admin admin] PATH_INFO: `/v3/users` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:38.002540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0be6a3-81df-41c6-8727-aa90713f289a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:38.004749 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0be6a3-81df-41c6-8727-aa90713f289a admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:38.066265 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1441/2868] 10.222.0.17 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:37 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:38.070372 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbbad7-bb3a-49ea-9139-08a0dca6f5b8 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:38.082221 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbbad7-bb3a-49ea-9139-08a0dca6f5b8 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:38.086914 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8706f-afc1-4676-b4a1-796a4f7b9c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:38.087575 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8706f-afc1-4676-b4a1-796a4f7b9c16 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:38.088546 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8706f-afc1-4676-b4a1-796a4f7b9c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:38.088835 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8706f-afc1-4676-b4a1-796a4f7b9c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:38.089669 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8706f-afc1-4676-b4a1-796a4f7b9c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:38.091966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8706f-afc1-4676-b4a1-796a4f7b9c16 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:38.102870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1428/2869] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:38.107182 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0be6a3-81df-41c6-8727-aa90713f289a admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:38.117789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0be6a3-81df-41c6-8727-aa90713f289a admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:38.122708 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47483fe5-fcfe-4f34-90be-a2a3e68ec540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:38.123681 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47483fe5-fcfe-4f34-90be-a2a3e68ec540 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:38.124026 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47483fe5-fcfe-4f34-90be-a2a3e68ec540 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:38.124363 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47483fe5-fcfe-4f34-90be-a2a3e68ec540 admin admin] PATH_INFO: `/v3/projects/d87cb3abc77d4d97800fbadf503d6a25/users/6e3a177619cf44179cb316e9eced68e0/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:38.141669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47483fe5-fcfe-4f34-90be-a2a3e68ec540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87cb3abc77d4d97800fbadf503d6a25, user_id=6e3a177619cf44179cb316e9eced68e0, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:38.144157 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47483fe5-fcfe-4f34-90be-a2a3e68ec540 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:38.249105 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1442/2870] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:38 2022] PUT /identity/v3/projects/d87cb3abc77d4d97800fbadf503d6a25/users/6e3a177619cf44179cb316e9eced68e0/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:38.253078 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8706f-afc1-4676-b4a1-796a4f7b9c16 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:38.263781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8706f-afc1-4676-b4a1-796a4f7b9c16 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:38.333534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d30c1-e3d1-4fb8-abf1-4813ec813013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:38.333534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d30c1-e3d1-4fb8-abf1-4813ec813013 admin admin] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:38.333534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d30c1-e3d1-4fb8-abf1-4813ec813013 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:38.333534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d30c1-e3d1-4fb8-abf1-4813ec813013 admin admin] PATH_INFO: `/v3/roles` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:38.333926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d30c1-e3d1-4fb8-abf1-4813ec813013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:38.333926 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d30c1-e3d1-4fb8-abf1-4813ec813013 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:38.341689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1429/2871] 10.222.0.17 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:38.345817 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47483fe5-fcfe-4f34-90be-a2a3e68ec540 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:38.357630 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47483fe5-fcfe-4f34-90be-a2a3e68ec540 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:38.363120 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4732f6d-890e-48a2-8f5f-52c7a4e9d6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:38.364025 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4732f6d-890e-48a2-8f5f-52c7a4e9d6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:38.364311 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4732f6d-890e-48a2-8f5f-52c7a4e9d6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:38.364593 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4732f6d-890e-48a2-8f5f-52c7a4e9d6e5 admin admin] PATH_INFO: `/v3/projects/d87cb3abc77d4d97800fbadf503d6a25/users/6e3a177619cf44179cb316e9eced68e0/roles/7a833c45603a4830b888e58e1232712a` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:38.366677 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4732f6d-890e-48a2-8f5f-52c7a4e9d6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87cb3abc77d4d97800fbadf503d6a25, user_id=6e3a177619cf44179cb316e9eced68e0, role_id=7a833c45603a4830b888e58e1232712a)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:38.369838 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4732f6d-890e-48a2-8f5f-52c7a4e9d6e5 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:38.382510 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1443/2872] 10.222.0.17 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:38 2022] PUT /identity/v3/projects/d87cb3abc77d4d97800fbadf503d6a25/users/6e3a177619cf44179cb316e9eced68e0/roles/7a833c45603a4830b888e58e1232712a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:43.928607 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52df4a-bce8-4b7a-a15d-a387b00e6747 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:43.929148 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52df4a-bce8-4b7a-a15d-a387b00e6747 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:43.929248 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52df4a-bce8-4b7a-a15d-a387b00e6747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:43.959743 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9f52df4a-bce8-4b7a-a15d-a387b00e6747 None None] MFA Rules not processed for user `6e3a177619cf44179cb316e9eced68e0`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:43.961125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9f52df4a-bce8-4b7a-a15d-a387b00e6747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:44.054435 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1430/2873] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:07:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:44.062763 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4732f6d-890e-48a2-8f5f-52c7a4e9d6e5 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:44.064555 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-f4732f6d-890e-48a2-8f5f-52c7a4e9d6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:44.143424 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4732f6d-890e-48a2-8f5f-52c7a4e9d6e5 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:44.149037 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb228bd-c4f1-43d1-ae8a-9b48a46bf82a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:44.149737 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb228bd-c4f1-43d1-ae8a-9b48a46bf82a service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:44.150024 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb228bd-c4f1-43d1-ae8a-9b48a46bf82a service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:44.150292 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb228bd-c4f1-43d1-ae8a-9b48a46bf82a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:44.219422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:44.222624 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb228bd-c4f1-43d1-ae8a-9b48a46bf82a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:44.224240 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb228bd-c4f1-43d1-ae8a-9b48a46bf82a service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:44.234540 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1444/2874] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:54.043674 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:54.276487 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=2 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:02.208027 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fcd9e-1d99-4137-a1b2-4280c8a7b67e None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:02.208622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fcd9e-1d99-4137-a1b2-4280c8a7b67e None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:02.208622 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fcd9e-1d99-4137-a1b2-4280c8a7b67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:02.230061 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-9c6fcd9e-1d99-4137-a1b2-4280c8a7b67e None None] MFA Rules not processed for user `8650e8e864f045c59cf38e7a50949b94`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:02.231328 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9c6fcd9e-1d99-4137-a1b2-4280c8a7b67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:02.243694 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1431/2875] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:08:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:02.550111 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb228bd-c4f1-43d1-ae8a-9b48a46bf82a service nova] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:02.551550 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-feb228bd-c4f1-43d1-ae8a-9b48a46bf82a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:02.608392 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb228bd-c4f1-43d1-ae8a-9b48a46bf82a service nova] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:02.612084 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b4e18-7ab4-4a7c-a657-41039b51ab88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:02.613207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b4e18-7ab4-4a7c-a657-41039b51ab88 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:02.613207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b4e18-7ab4-4a7c-a657-41039b51ab88 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:02.613207 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b4e18-7ab4-4a7c-a657-41039b51ab88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:02.615316 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-903b4e18-7ab4-4a7c-a657-41039b51ab88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:02.624850 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b4e18-7ab4-4a7c-a657-41039b51ab88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:02.626269 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b4e18-7ab4-4a7c-a657-41039b51ab88 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:02.634355 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1445/2876] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:12.224528 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:12.630597 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=26 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:20.895305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fcd9e-1d99-4137-a1b2-4280c8a7b67e None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:20.905351 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fcd9e-1d99-4137-a1b2-4280c8a7b67e None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:20.908951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c63671-8786-4b8b-adf6-40eefc43dfe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:20.910269 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c63671-8786-4b8b-adf6-40eefc43dfe2 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:20.910384 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c63671-8786-4b8b-adf6-40eefc43dfe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:20.910676 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c63671-8786-4b8b-adf6-40eefc43dfe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:20.920079 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c63671-8786-4b8b-adf6-40eefc43dfe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:20.921775 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c63671-8786-4b8b-adf6-40eefc43dfe2 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:20.931951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1432/2877] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:30.927643 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=6 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:49.465434 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b4e18-7ab4-4a7c-a657-41039b51ab88 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:49.474027 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b4e18-7ab4-4a7c-a657-41039b51ab88 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:49.478368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91d0c2-ecf2-4cb8-8378-3e77672feab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:49.479031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91d0c2-ecf2-4cb8-8378-3e77672feab7 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:49.479312 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91d0c2-ecf2-4cb8-8378-3e77672feab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:49.479585 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91d0c2-ecf2-4cb8-8378-3e77672feab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:49.481519 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-4a91d0c2-ecf2-4cb8-8378-3e77672feab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:49.495715 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91d0c2-ecf2-4cb8-8378-3e77672feab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:49.497126 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91d0c2-ecf2-4cb8-8378-3e77672feab7 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:49.508046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1446/2878] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:55.518195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c63671-8786-4b8b-adf6-40eefc43dfe2 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:55.527758 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c63671-8786-4b8b-adf6-40eefc43dfe2 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:55.531305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8198e-9d40-4c34-ab3f-6a9290b02a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:55.531937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8198e-9d40-4c34-ab3f-6a9290b02a9c service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:55.532215 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8198e-9d40-4c34-ab3f-6a9290b02a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:55.532486 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8198e-9d40-4c34-ab3f-6a9290b02a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:55.534233 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2ac8198e-9d40-4c34-ab3f-6a9290b02a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:55.547932 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8198e-9d40-4c34-ab3f-6a9290b02a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:55.549342 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8198e-9d40-4c34-ab3f-6a9290b02a9c service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:55.560612 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1433/2879] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:57.948984 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70d3f5-3958-4ae7-93a6-e0cc980aa46c None None] REQUEST_METHOD: `POST` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:57.949475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70d3f5-3958-4ae7-93a6-e0cc980aa46c None None] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:57.949578 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70d3f5-3958-4ae7-93a6-e0cc980aa46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:57.967192 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.auth.core [None req-bf70d3f5-3958-4ae7-93a6-e0cc980aa46c None None] MFA Rules not processed for user `56d11b3ed0474760a10e6f94df838297`. Rule list: `[]` (Enabled: `True`). {{(pid=91980) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:57.968393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bf70d3f5-3958-4ae7-93a6-e0cc980aa46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:58.022115 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1447/2880] 10.222.0.17 () {58 vars in 967 bytes} [Mon Jul 25 06:08:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:58.064581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8198e-9d40-4c34-ab3f-6a9290b02a9c service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:58.064581 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-2ac8198e-9d40-4c34-ab3f-6a9290b02a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:58.131778 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8198e-9d40-4c34-ab3f-6a9290b02a9c service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:58.136755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44bf5-1f25-4447-98d0-583611397112 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c359fc0168141fb8da3f953884fd2e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:58.137393 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44bf5-1f25-4447-98d0-583611397112 service swift] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:58.137672 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44bf5-1f25-4447-98d0-583611397112 service swift] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:58.137975 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44bf5-1f25-4447-98d0-583611397112 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:58.145911 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44bf5-1f25-4447-98d0-583611397112 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:58.147496 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44bf5-1f25-4447-98d0-583611397112 service swift] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:58.156058 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1434/2881] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Jul 25 06:08:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:09:06.575839 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70d3f5-3958-4ae7-93a6-e0cc980aa46c None None] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:09:06.576373 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-bf70d3f5-3958-4ae7-93a6-e0cc980aa46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:09:06.641891 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70d3f5-3958-4ae7-93a6-e0cc980aa46c None None] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:09:06.646264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:09:06.647164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:09:06.647445 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:09:06.647718 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] PATH_INFO: `/v3/users/a01a4b9868714295a1000e0a5a81143b` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:09:06.649494 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a01a4b9868714295a1000e0a5a81143b)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:09:06.651422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:09:06.865922 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a01a4b9868714295a1000e0a5a81143b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:06.873331 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a01a4b9868714295a1000e0a5a81143b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:06.874079 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a01a4b9868714295a1000e0a5a81143b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:06.950501 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a01a4b9868714295a1000e0a5a81143b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:06.951114 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a01a4b9868714295a1000e0a5a81143b'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:06.968955 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1448/2882] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:09:06 2022] DELETE /identity/v3/users/a01a4b9868714295a1000e0a5a81143b => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:09:07.226368 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44bf5-1f25-4447-98d0-583611397112 service swift] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:09:07.242913 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44bf5-1f25-4447-98d0-583611397112 service swift] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:09:07.307710 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:09:07.308592 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:09:07.308853 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:09:07.309122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] PATH_INFO: `/v3/projects/a39f82493df2479395de88147e2f8292` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:09:07.310311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a39f82493df2479395de88147e2f8292)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:09:07.312245 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:09:07.395020 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a39f82493df2479395de88147e2f8292', 'request_id': 'req-478a054f-fb22-433f-8801-83da01e602d7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:07.401026 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a39f82493df2479395de88147e2f8292', 'request_id': 'req-478a054f-fb22-433f-8801-83da01e602d7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:07.406850 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a39f82493df2479395de88147e2f8292', 'request_id': 'req-478a054f-fb22-433f-8801-83da01e602d7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:07.474338 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] The token cache is being invalidate because project a39f82493df2479395de88147e2f8292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:09:07.474601 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:07.475258 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:07.477413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1435/2883] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:09:07 2022] DELETE /identity/v3/projects/a39f82493df2479395de88147e2f8292 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:09:07.481709 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:09:07.483458 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:09:07.497158 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2aa21e-b1cf-4415-8539-b6287e88f512 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:09:07.504905 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:09:07.552081 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:09:07.552951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:09:07.553231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:09:07.553505 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] PATH_INFO: `/v3/users/a7d69a23418e491f883936f409067f26` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:09:07.555368 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d69a23418e491f883936f409067f26)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:09:07.557266 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:09:07.695886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d69a23418e491f883936f409067f26'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:07.704046 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d69a23418e491f883936f409067f26'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:07.704837 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d69a23418e491f883936f409067f26'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:07.750341 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d69a23418e491f883936f409067f26'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:07.750989 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d69a23418e491f883936f409067f26'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:07.768733 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1449/2884] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:09:07 2022] DELETE /identity/v3/users/a7d69a23418e491f883936f409067f26 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:09:08.043465 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:09:08.046871 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:09:08.058937 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a054f-fb22-433f-8801-83da01e602d7 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:09:08.062641 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:09:08.116629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:09:08.117489 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:09:08.117763 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:09:08.118032 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] PATH_INFO: `/v3/projects/61334b1f57eb4f599673de8b21300e1e` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:09:08.119223 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61334b1f57eb4f599673de8b21300e1e)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:09:08.121190 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:09:08.247168 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61334b1f57eb4f599673de8b21300e1e', 'request_id': 'req-ba63b8ac-ac02-4d61-ac60-42a8901891e3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:08.253366 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61334b1f57eb4f599673de8b21300e1e', 'request_id': 'req-ba63b8ac-ac02-4d61-ac60-42a8901891e3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:08.259502 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61334b1f57eb4f599673de8b21300e1e', 'request_id': 'req-ba63b8ac-ac02-4d61-ac60-42a8901891e3'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:08.326658 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] The token cache is being invalidate because project 61334b1f57eb4f599673de8b21300e1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:09:08.326951 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:08.327585 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ba63b8ac-ac02-4d61-ac60-42a8901891e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:08.329185 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1436/2885] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:09:08 2022] DELETE /identity/v3/projects/61334b1f57eb4f599673de8b21300e1e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:09:17.765129 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:09:18.323434 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:09:48.221840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:09:48.223350 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:09:48.281015 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7b0af-a9b2-47e6-91f8-2353e255cc9c admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:09:48.333233 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318ed29-4db9-4c0d-9bcc-c720043807e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:09:48.333884 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318ed29-4db9-4c0d-9bcc-c720043807e1 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:09:48.334166 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318ed29-4db9-4c0d-9bcc-c720043807e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:09:48.334442 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318ed29-4db9-4c0d-9bcc-c720043807e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:09:48.336262 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6318ed29-4db9-4c0d-9bcc-c720043807e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:09:48.349561 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318ed29-4db9-4c0d-9bcc-c720043807e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:09:48.351044 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318ed29-4db9-4c0d-9bcc-c720043807e1 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:09:48.359846 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1450/2886] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:09:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:09:58.356241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=46 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:10:23.175125 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd48d60-88a4-4e69-8e33-95ba92788b86 None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:23.175610 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd48d60-88a4-4e69-8e33-95ba92788b86 None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:23.175713 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd48d60-88a4-4e69-8e33-95ba92788b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:23.193689 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-3fd48d60-88a4-4e69-8e33-95ba92788b86 None None] MFA Rules not processed for user `8650e8e864f045c59cf38e7a50949b94`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:10:23.194925 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3fd48d60-88a4-4e69-8e33-95ba92788b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:23.248028 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1437/2887] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:10:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:10:23.549828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318ed29-4db9-4c0d-9bcc-c720043807e1 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:23.551231 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-6318ed29-4db9-4c0d-9bcc-c720043807e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:23.561691 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318ed29-4db9-4c0d-9bcc-c720043807e1 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:23.565306 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a687dc0-71a0-494f-8db2-397acbf7571b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:23.565936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a687dc0-71a0-494f-8db2-397acbf7571b service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:23.566230 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a687dc0-71a0-494f-8db2-397acbf7571b service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:23.566528 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a687dc0-71a0-494f-8db2-397acbf7571b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:23.568229 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9a687dc0-71a0-494f-8db2-397acbf7571b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:23.578053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a687dc0-71a0-494f-8db2-397acbf7571b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:23.579491 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a687dc0-71a0-494f-8db2-397acbf7571b service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:23.587493 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1451/2888] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:24.284548 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd48d60-88a4-4e69-8e33-95ba92788b86 None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:24.286014 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3fd48d60-88a4-4e69-8e33-95ba92788b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:24.358213 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd48d60-88a4-4e69-8e33-95ba92788b86 None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:24.363195 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a444278-942e-4f39-8ddf-b539c3014c79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:24.363843 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a444278-942e-4f39-8ddf-b539c3014c79 service placement] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:24.364122 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a444278-942e-4f39-8ddf-b539c3014c79 service placement] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:24.364395 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a444278-942e-4f39-8ddf-b539c3014c79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:24.366113 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4a444278-942e-4f39-8ddf-b539c3014c79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:24.377706 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a444278-942e-4f39-8ddf-b539c3014c79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:24.379302 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a444278-942e-4f39-8ddf-b539c3014c79 service placement] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:24.389068 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1438/2889] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:33.583854 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:10:34.384801 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=57 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:10:38.993779 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a687dc0-71a0-494f-8db2-397acbf7571b service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:38.995568 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-9a687dc0-71a0-494f-8db2-397acbf7571b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:39.068704 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a687dc0-71a0-494f-8db2-397acbf7571b service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:39.076402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9615f1f-e810-4ae8-a3c2-a70858f3067c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:39.076402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9615f1f-e810-4ae8-a3c2-a70858f3067c service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:39.076402 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9615f1f-e810-4ae8-a3c2-a70858f3067c service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:39.076789 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9615f1f-e810-4ae8-a3c2-a70858f3067c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:39.078414 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d9615f1f-e810-4ae8-a3c2-a70858f3067c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:39.091273 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9615f1f-e810-4ae8-a3c2-a70858f3067c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:39.092859 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9615f1f-e810-4ae8-a3c2-a70858f3067c service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:39.103886 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1452/2890] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:43.902879 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a444278-942e-4f39-8ddf-b539c3014c79 service placement] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:43.905545 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4a444278-942e-4f39-8ddf-b539c3014c79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:43.921117 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a444278-942e-4f39-8ddf-b539c3014c79 service placement] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:43.926928 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e24ea2-d4ae-42d2-99f9-a91691d0846f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:43.927588 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e24ea2-d4ae-42d2-99f9-a91691d0846f service nova] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:43.927870 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e24ea2-d4ae-42d2-99f9-a91691d0846f service nova] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:43.928143 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e24ea2-d4ae-42d2-99f9-a91691d0846f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:43.929966 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-84e24ea2-d4ae-42d2-99f9-a91691d0846f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:43.942498 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e24ea2-d4ae-42d2-99f9-a91691d0846f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:43.944070 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e24ea2-d4ae-42d2-99f9-a91691d0846f service nova] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:43.955276 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1439/2891] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:49.099951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=34 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:10:53.950092 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=6 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:11:47.477513 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9615f1f-e810-4ae8-a3c2-a70858f3067c service nova] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:11:47.478967 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d9615f1f-e810-4ae8-a3c2-a70858f3067c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:11:47.504656 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9615f1f-e810-4ae8-a3c2-a70858f3067c service nova] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:11:47.517234 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7fb01-997c-4cfd-8a9e-72ec75e01776 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:11:47.517908 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7fb01-997c-4cfd-8a9e-72ec75e01776 service placement] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:11:47.518193 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7fb01-997c-4cfd-8a9e-72ec75e01776 service placement] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:11:47.518488 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7fb01-997c-4cfd-8a9e-72ec75e01776 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:11:47.520851 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-e0d7fb01-997c-4cfd-8a9e-72ec75e01776 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:11:47.536536 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7fb01-997c-4cfd-8a9e-72ec75e01776 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:11:47.538124 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7fb01-997c-4cfd-8a9e-72ec75e01776 service placement] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:11:47.548590 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1453/2892] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:11:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:11:57.543894 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:12:18.445830 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e980a-6e6b-42bd-b64f-a20855a0980b None None] REQUEST_METHOD: `POST` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:18.446359 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e980a-6e6b-42bd-b64f-a20855a0980b None None] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:18.446558 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e980a-6e6b-42bd-b64f-a20855a0980b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:18.464030 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.auth.core [None req-4d9e980a-6e6b-42bd-b64f-a20855a0980b None None] MFA Rules not processed for user `8650e8e864f045c59cf38e7a50949b94`. Rule list: `[]` (Enabled: `True`). {{(pid=91981) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:12:18.465227 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-4d9e980a-6e6b-42bd-b64f-a20855a0980b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:18.472138 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1440/2893] 10.222.0.17 () {58 vars in 957 bytes} [Mon Jul 25 06:12:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:12:23.165178 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7fb01-997c-4cfd-8a9e-72ec75e01776 service placement] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:23.181589 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7fb01-997c-4cfd-8a9e-72ec75e01776 service placement] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:23.205615 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc52ac-27b6-427e-9376-cb5d81edea2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '067b827625e0427294b93b027597a5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:23.206298 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc52ac-27b6-427e-9376-cb5d81edea2b service nova] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:23.206652 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc52ac-27b6-427e-9376-cb5d81edea2b service nova] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:23.207358 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc52ac-27b6-427e-9376-cb5d81edea2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:23.210566 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-adfc52ac-27b6-427e-9376-cb5d81edea2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:23.224951 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc52ac-27b6-427e-9376-cb5d81edea2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:23.226633 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc52ac-27b6-427e-9376-cb5d81edea2b service nova] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:23.239870 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1454/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:12:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:12:28.458884 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=3 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:12:33.233785 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:14:19.722992 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e980a-6e6b-42bd-b64f-a20855a0980b None None] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:19.732998 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e980a-6e6b-42bd-b64f-a20855a0980b None None] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:19.736632 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24eb42-93fb-493c-b913-c6f418c84088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:19.737781 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24eb42-93fb-493c-b913-c6f418c84088 service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:19.738065 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24eb42-93fb-493c-b913-c6f418c84088 service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:19.738340 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24eb42-93fb-493c-b913-c6f418c84088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:19.740162 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3c24eb42-93fb-493c-b913-c6f418c84088 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:19.751311 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24eb42-93fb-493c-b913-c6f418c84088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:19.753319 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24eb42-93fb-493c-b913-c6f418c84088 service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:19.762454 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1441/2895] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:14:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:14:21.517778 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc52ac-27b6-427e-9376-cb5d81edea2b service nova] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:21.529754 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc52ac-27b6-427e-9376-cb5d81edea2b service nova] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:21.534345 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279c61c-e675-4f92-aab2-cd51b7bb0637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:21.535241 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c61c-e675-4f92-aab2-cd51b7bb0637 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:21.535702 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c61c-e675-4f92-aab2-cd51b7bb0637 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:21.536164 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c61c-e675-4f92-aab2-cd51b7bb0637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:21.538439 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3279c61c-e675-4f92-aab2-cd51b7bb0637 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:21.553940 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279c61c-e675-4f92-aab2-cd51b7bb0637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:21.555777 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279c61c-e675-4f92-aab2-cd51b7bb0637 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:21.568309 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1455/2896] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:14:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:14:22.748154 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24eb42-93fb-493c-b913-c6f418c84088 service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:22.749755 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-3c24eb42-93fb-493c-b913-c6f418c84088 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:22.761667 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24eb42-93fb-493c-b913-c6f418c84088 service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:22.765277 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df0b30-9fc2-4acb-ad8d-1e56bd927cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:22.765921 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df0b30-9fc2-4acb-ad8d-1e56bd927cab service neutron] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:22.766206 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df0b30-9fc2-4acb-ad8d-1e56bd927cab service neutron] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:22.766514 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df0b30-9fc2-4acb-ad8d-1e56bd927cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:22.768303 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-00df0b30-9fc2-4acb-ad8d-1e56bd927cab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:22.782329 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df0b30-9fc2-4acb-ad8d-1e56bd927cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:22.783919 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df0b30-9fc2-4acb-ad8d-1e56bd927cab service neutron] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:22.798055 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1442/2897] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:14:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:14:31.562682 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=6 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:14:32.792387 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=12 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:14:34.833935 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279c61c-e675-4f92-aab2-cd51b7bb0637 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:34.842647 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279c61c-e675-4f92-aab2-cd51b7bb0637 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:34.846264 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1135-e141-41f4-b129-e95ec7875fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecd61347f8874b89aada19772ec56179', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:34.846946 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1135-e141-41f4-b129-e95ec7875fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:34.847226 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1135-e141-41f4-b129-e95ec7875fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:34.847515 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1135-e141-41f4-b129-e95ec7875fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:34.849258 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8f8a1135-e141-41f4-b129-e95ec7875fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:34.917794 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a1135-e141-41f4-b129-e95ec7875fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:34.919296 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a1135-e141-41f4-b129-e95ec7875fb6 service neutron] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:34.930031 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1456/2898] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:14:38.364505 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df0b30-9fc2-4acb-ad8d-1e56bd927cab service neutron] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:38.376551 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df0b30-9fc2-4acb-ad8d-1e56bd927cab service neutron] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:38.388044 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8c798-4758-4b92-9cea-ac267dab7b9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:38.388712 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8c798-4758-4b92-9cea-ac267dab7b9e service placement] REQUEST_METHOD: `GET` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:38.388994 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8c798-4758-4b92-9cea-ac267dab7b9e service placement] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:38.389272 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8c798-4758-4b92-9cea-ac267dab7b9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:38.391237 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-9cc8c798-4758-4b92-9cea-ac267dab7b9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:38.404136 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8c798-4758-4b92-9cea-ac267dab7b9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:38.405732 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8c798-4758-4b92-9cea-ac267dab7b9e service placement] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:38.416629 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1443/2899] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:14:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:14:42.437840 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1135-e141-41f4-b129-e95ec7875fb6 service neutron] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:42.442828 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-8f8a1135-e141-41f4-b129-e95ec7875fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:42.455160 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1135-e141-41f4-b129-e95ec7875fb6 service neutron] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:42.459825 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:42.460732 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:42.461018 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:42.461295 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] PATH_INFO: `/v3/users/8650e8e864f045c59cf38e7a50949b94` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:42.462572 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8650e8e864f045c59cf38e7a50949b94)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:42.464461 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:42.669407 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8650e8e864f045c59cf38e7a50949b94'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:42.677422 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8650e8e864f045c59cf38e7a50949b94'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:42.678074 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8650e8e864f045c59cf38e7a50949b94'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:42.770012 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8650e8e864f045c59cf38e7a50949b94'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:42.770692 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8650e8e864f045c59cf38e7a50949b94'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:42.789739 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1457/2900] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:14:42 2022] DELETE /identity/v3/users/8650e8e864f045c59cf38e7a50949b94 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:14:43.053013 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8c798-4758-4b92-9cea-ac267dab7b9e service placement] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:43.066737 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8c798-4758-4b92-9cea-ac267dab7b9e service placement] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:43.125534 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:43.126402 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:43.126719 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:43.126999 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] PATH_INFO: `/v3/projects/5eef98733468492383e255f9045cd025` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:43.128213 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eef98733468492383e255f9045cd025)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:43.130121 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:43.240081 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eef98733468492383e255f9045cd025', 'request_id': 'req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:43.246256 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eef98733468492383e255f9045cd025', 'request_id': 'req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:43.251820 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eef98733468492383e255f9045cd025', 'request_id': 'req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:43.353779 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] The token cache is being invalidate because project 5eef98733468492383e255f9045cd025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:14:43.357704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:43.357704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:43.357704 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1444/2901] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:14:43 2022] DELETE /identity/v3/projects/5eef98733468492383e255f9045cd025 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:14:43.360969 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:43.362869 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:43.375755 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3f94-14b8-485f-b914-bcf4b5e41804 admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:43.432975 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:43.433866 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:43.434203 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:43.434520 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] PATH_INFO: `/v3/users/6e3a177619cf44179cb316e9eced68e0` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:43.435741 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3a177619cf44179cb316e9eced68e0)` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:43.437713 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:43.443936 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:14:43.678504 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3a177619cf44179cb316e9eced68e0'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:43.684669 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3a177619cf44179cb316e9eced68e0'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:43.685393 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3a177619cf44179cb316e9eced68e0'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:43.792924 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3a177619cf44179cb316e9eced68e0'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:43.794956 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.notifications [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3a177619cf44179cb316e9eced68e0'} {{(pid=91980) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:43.812308 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1458/2902] 10.222.0.17 () {58 vars in 1223 bytes} [Mon Jul 25 06:14:43 2022] DELETE /identity/v3/users/6e3a177619cf44179cb316e9eced68e0 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:14:44.126431 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] Authenticating user token {{(pid=91981) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:44.134772 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.fernet_utils [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91981) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:44.142826 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce179c4c-ea77-4e94-910f-8eb7f958b0e0 admin admin] Validating token access rules against request {{(pid=91981) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:44.207049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16be6f6629504bf5b97a9530841ecf24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '799d47101892496a8efa7ee2b0f5f24a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91981) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:44.207049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:44.207049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:44.207049 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] PATH_INFO: `/v3/projects/d87cb3abc77d4d97800fbadf503d6a25` {{(pid=91981) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:44.207552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d87cb3abc77d4d97800fbadf503d6a25)` {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:44.207552 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] RBAC: Authorization granted {{(pid=91981) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:44.228776 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:14:44.370414 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd87cb3abc77d4d97800fbadf503d6a25', 'request_id': 'req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:44.376413 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd87cb3abc77d4d97800fbadf503d6a25', 'request_id': 'req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:44.382201 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd87cb3abc77d4d97800fbadf503d6a25', 'request_id': 'req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7'} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:44.508806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] The token cache is being invalidate because project d87cb3abc77d4d97800fbadf503d6a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91981) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:14:44.508806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:44.508806 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG keystone.notifications [None req-c69ad993-3dd5-426c-a77e-8f5cb6fa7af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91981) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:44.509906 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: [pid: 91981|app: 0|req: 1445/2903] 10.222.0.17 () {58 vars in 1232 bytes} [Mon Jul 25 06:14:44 2022] DELETE /identity/v3/projects/d87cb3abc77d4d97800fbadf503d6a25 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:14:53.838579 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:14:54.504305 dv-cinder-850831-1-iscsi devstack@keystone.service[91981]: DEBUG dbcounter [-] [91981] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91981) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:16:51.452053 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] Authenticating user token {{(pid=91980) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:16:51.461972 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:16:51.604141 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7c1a2-4ded-48ab-89c6-b632ad956acb admin admin] Validating token access rules against request {{(pid=91980) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:16:51.662151 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ffa13-8978-4ad7-978f-fa8c15236069 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbfcf2e915c049f08a9e41c555401a6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32cbea18a5314504bb39fcd5fdbaa3c3', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91980) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:16:51.662904 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ffa13-8978-4ad7-978f-fa8c15236069 service placement] REQUEST_METHOD: `GET` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:16:51.663191 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ffa13-8978-4ad7-978f-fa8c15236069 service placement] SCRIPT_NAME: `/identity` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:16:51.663475 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ffa13-8978-4ad7-978f-fa8c15236069 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91980) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:16:51.665927 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.fernet_utils [None req-d60ffa13-8978-4ad7-978f-fa8c15236069 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91980) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:16:51.677890 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ffa13-8978-4ad7-978f-fa8c15236069 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:16:51.679538 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ffa13-8978-4ad7-978f-fa8c15236069 service placement] RBAC: Authorization granted {{(pid=91980) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:16:51.689634 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: [pid: 91980|app: 0|req: 1459/2904] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Jul 25 06:16:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:17:01.685259 dv-cinder-850831-1-iscsi devstack@keystone.service[91980]: DEBUG dbcounter [-] [91980] Writing DB stats keystone:SELECT=89 {{(pid=91980) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}